#  Language Agnostic Data-Driven Inverse Text Normalization

###### Abstract

With the emergence of automatic speech recognition (ASR) models, converting
the spoken form text (from ASR) to the written form is in urgent need. This
inverse text normalization (ITN) problem attracts the attention of researchers
from various fields. Recently, several works show that data-driven ITN methods
can output high-quality written form text. Due to the scarcity of labeled
spoken-written datasets, the studies on non-English data-driven ITN are quite
limited. In this work, we propose a language-agnostic data-driven ITN
framework to fill this gap. Specifically, we leverage the data augmentation in
conjunction with neural machine translated data for low resource languages.
Moreover, we design an evaluation method for language agnostic ITN model when
only English data is available. Our empirical evaluation shows this language
agnostic modeling approach is effective for low resource languages while
preserving the performance for high resource languages.

Index Terms—  Inverse text normalization, Multilingual, Language-agnostic

##  1 Introduction

Recent progress in automatic speech recognition (ASR) technologies brings
world-wide adoption to use voice as a source of input to interact and
communicate with digital environments. Moreover, ASR systems are now expanding
language horizons to bring same or better experience to native language
speaker around the world. For communication and better readability of spoken
form output from an ASR system, it needs to be paired with an inverse text
normalization (ITN) system to produce corresponding written form texts.

Converting spoken form texts to written form is not an easy task. For example,
the spoken form twenty twenty can be written as (a) 2020 to express the
year/number, (b) 20:20 for time, (c) 20/20 for eye-vision, (d) 20-20 for a
score in game or a cricket match. Taking another example of ten to twelve,
which can be written as (a) 10-12 as a cardinal number range, (b) 10:00-12:00
as time range, (c) 11:50 am/pm as a time instance, etc. It is evident from
these examples that the speaker contexts could help in determining the correct
written form which is hard to abide by a rule-based system. Therefore data-
driven based systems are gaining attention of researchers [1, 2, 3, 4] to
improve ITN systems, dub as DD-ITN models.

Dataset of spoken-written text pairs are used to train these neural network
based DD-ITN models. Obtaining spoken-written text pairs that covers diverse
ITN entities such as cardinals, ordinals, date-time, money, fractions,
decimals, address, metrics, email, URL, abbreviation etc. is hard but doable
for high-resource languages such as English.  In addition, the written
representation of same entity can vary across languages, e.g., 3:30 pm is
commonly represented as 15h30 in French. The challenge grows multi-fold while
expanding to more languages, especially low-resource ones where it is hard to
obtain spoken-written text pairs dataset.

We discovered that separating the ITN component from the end-to-end ASR system
allows for independent performance improvements using a relatively large text
only dataset. Moreover, our in-house on-device spoken-form ASR systems have
achieved a significant reduction in word-error-rate (WER) of $>$10%, a
reduction in real-time-factor (RTF) of $>$8%, and lower memory consumption.
This has led us to propose using the ITN as a low-latency post-processing
service. In this study, we aim to internationalize (i18n) and expand the
language capabilities of ITN models by addressing the problem of data scarcity
and presenting a unified, language-agnostic ITN model that supports multiple
languages. The main contributions of this work are as follows:

  * •

We propose a text normalization method for English that transforms written
form texts to spoken form texts. Unlike conventional text normalization
system, our data augmentation system generates more possible variants of
spoken forms; which can help  build robust ITN system.

  * •

We propose to apply neural machine translation for internationalization of the
ITN models, which can be considered as a knowledge distillation approach. We
use neural machine translation on English spoken-written text pairs to
generate spoken-written pairs on target languages; and it helps ITN expanding
to more languages.

  * •

We present a language agnostic data-driven ITN model that supports inverse
normalization of spoken form texts for 12 languages.  We also present a study
of system design choices in our experiment section.

Fig. 1: Multilingual data generation using rule based text normalization
system and machine translation model.

##  2 Methodology

###  2.1 Data Augmentation

To train a language agnostic ITN system, we need spoken-written text pairs for
every language. However, there are not much public available resources having
spoken-written text pairs in English, not to mention other languages. Hence,
we propose a two-step data augmentation method to generate spoken-written text
pairs for multiple languages. First, we generate spoken form texts from highly
available written text resources in English. Then, we use machine translation
to generate text pairs for other languages.

####  2.1.1 Enhanced Text Normalization

Traditional text normalization (TN) used in text-to-speech (TTS) systems
generates fixed variation of spoken forms (often with rule-based approach)
that conforms to the verbalizer standard for any given written text with TN-
ITN entities. As mentioned previously, spoken forms lack full information
about the subject; hence, it is important to cover more spoken variations and
alternatives. With  abundant written text resources in hand,  we apply our
specialized text normalization augmentation system that generates many
possible spoken variations; statistically 22$\times$ more diverse than
conventional TN system. See Table 1 for a few examples and we recommend
readers to check the Appendix Sec. 6.1 in our extended version of paper for
details [5].

Table 1: Examples of generated spoken form using conventional TN system and our data augmentation system Written Text | Spoken Text from | Spoken Text from
---|---|---
Input | Conventional TN | Enhanced TN System
6:15 am |  | six fifteen a m
---
| six fifteen a m
---
six fifteen in the morning
six fifteen
six past fifteen a m
quarter past six a m
quarter past six morning
six and quarter a m
$1.20 |  | one dollar and twenty cents
---
| one dollar and twenty cents
---
one dollar twenty cents
one dollar two zero cents
one point two zero dollars
a dollar twenty cents

####  2.1.2 Multilingual spoken-written text pairs

We propose using neural machine translation (NMT) models to generate spoken-
written text pairs in target languages for which we do not have adequate
pairs. However, we have found that the outputs of the NMT model do not always
meet our criteria for quality. To ensure the quality of our spoken-written
text pairs, we have implemented the following measures: (a) Spoken/Written
Mismatch: Discarding translated texts that have mismatches between written and
spoken forms, (b) Word Error Rate (WER): Strictly adhering to the Word Error
Rate (WER) metric for selecting non-ITN text segments, (c) Target Language
Conformity: Ensuring conformity between the source and target languages, and
filtering out any malformed or incorrect translations with input from
linguists, like 801 $\rightarrow$ eight o one [en] $\neq$ otto o uno [it],
etc. We provide translation accuracy in Table 6, the pictorial position of
translation module in the pipeline in Fig. 1, and a few translation examples
in Table 2 for reference.

Table 2: Examples of data augmentation with machine translation models for French [fr], Italian [it], Spanish [es]. Form | Text in English | Translated text
---|---|---
spoken |  | Historical average for
---
January is thirty one
degrees.
| La moyenne historique de janvier
---
est de trente et un degrés [fr]
La media storica di gennaio
è di trentuno gradi. [it]
La media histórica de enero
es de treinta y un grados. [es]
written |  | Historical average for
---
January is 31 degrees.
| La moyenne historique pour janvier
---
est de 31 degrés. [fr]
La media storica di gennaio
è di 31 gradi. [it]
La media histórica de enero
es de 31 grados. [es]

###  2.2 Model Architecture

For our ITN task, it can be seen as a sequence-to-sequence (Seq2Seq) problem,
which turns a sequence in one domain (e.g., spoken domain) to a sequence in
another domain (e.g., written domain). For this Seq2Seq problem, we employ the
Encoder-Decoder architecture (Fig. 2) to solve it. More specifically, two
types of Encoder-Decoder model are investigated in this work: the LSTM-based
Seq2Seq model and the Transformer model.

Fig. 2: Encoder-Decoder model architecture for ITN.

With the sequence data, a natural idea is to use the recurrent neural network
(RNN), and Long Short-Term Memory (LSTM) [6] is the first choice among
Recurrent Neural Networks (RNNs). In our LSTM-based Seq2Seq model, we utilize
the LSTM in both the encoder and decoder. In order to encode the input
sequence better, we employ bi-directional encoder to consider both the left
context and right context in the sequence. Also, in the decoder side, we use
attention mechanism to derive a context vector that captures the relevant
source-side (encoder-side) information to help prediction. Initially, this
type of Seq2Seq model is used for translation task. For more details about the
model, we refer readers to paper [7].

For the Transformer model, we employ the original model from [8]. Its encoder
and decoder are composed of stacked modules (or layers), and each module
mainly consists of multi-head attention and feed forward networks. The
attention mechanism will take the whole sequence into account by learning
weights for input tokens in the encoder. In the decoder, the masked attention
mechanism is applied to predict the next token based on the previous tokens.

For fair comparison, we experiment with both models with similar parameter
sizes. The details of the model parameter can be found in Table 3.

Table 3: Model Parameters. | LSTM | Transformer
---|---|---
No. of Parameter | 19.98M | 19.67M
Encoder Layer | 2 | 4
Decoder Layer | 2 | 4
Hidden Size | 256 | 256
Attention Head | n/a | 8

##  3 Experiments

We select 12 languages based on their richness of resources and their writing
script, as shown in Table 4. For example,  we see Russian and Kazakh share the
same Cyrillic script where the latter is a low-resource language. For the
model input/output text tokenization, we use SentencePiece tokenizer model
(SPM)[9] with a vocabulary size of 20,000. Our ablation study with varying SPM
vocabulary sizes (omitted due to space constraints) shows little performance
improvement beyond that. To note, unless specified in experiment results, LSTM
based Seq2Seq architecture is our default ITN model as described in Section
2.2. As choice of NMT models for our experiments, we use (a) Opus-MT [10] with
EasyNMT library support, (b) MetaNMT, an in-house version of NMT with
comparable performance to NLLB [11]. Also, unless specified, we use MetaNMT
for data augmentation as it has better BLEU scores (see respective papers) and
ITN performance impact (see Table 8).

Table 4: 12 languages selected for experiments. Resources | Latin Script | Non-Latin Script
---|---|---
High |  Italian [it], French [fr], Spanish [es], |  Russian [ru], Greek [el]
English [en], Turkish [tr], German [de] |
Low |  Icelandic [is], Afrikaans [af] |  Tamil [ta], Kazakh [kk]

###  3.1 Dataset

We use the OpenSubtitles[12] and TED2020[13] datasets from
OPUS111https://opus.nlpl.eu/ as our training data. To be specific, only the
written English texts from OPUS are used in our method. The spoken-written
pairs for training are generated using the pipeline in Fig. 1.

We evaluate our models on two datasets: (a) Dictation testset: Human annotated
6,810 spoken-written conversational text pairs in English containing diverse
ITN entities in mixed proportions. We apply the Case B strategy described in
Fig 3 and Section 3.2 to generate evaluation data for non-English languages.
(b) Caption testset: Mostly containing mathematical expressions, measures,
metrics, phone numbers collected from audios with uploaded caption. This
dataset has [en]:22332, [es]:21216, [fr]:27300, [it]:14939, [de]:5960 spoken-
written text pairs for respective languages.

###  3.2 ITN Evaluation

For the evaluation of our ITN model, we  mainly focus on the numerical
entities in the text. The evaluation is straightforward when ground-truth
target language spoken-written pairs are available. But the lacking of proper
human annotated spoken-written pairs for mid/low resource languages motivates
us to propose an approach to measure the model performance on the numerical
entities shown in Fig. 3.

####  3.2.1 ITN normalized accuracy

In Fig. 3, Case A shows the evaluation approach when target language spoken-
written pair dataset is available.  We apply Case B where target language pair
dataset is not available, we translate the spoken form text from English human
annotated dataset to target language. We then apply the trained ITN model to
obtain written form text in target language. Thereafter, we verify the digit
in the written form is the same as the original one in English. If they are
the same, we count it as a correct instance, if not, we count it as a
incorrect one. To note,  translation models may output written form in target
language for spoken source input, we discard them and only evaluate the ITN
model on correctly translated spoken form texts. We use normalized accuracy to
measure the performance of our ITN model expressed as the fraction of correct
prediction over all the ITN entities.

Fig. 3: ITN evaluation strategy with case A: Target language evaluation
dataset is available; Case B: Target language evaluation dataset is not
available. We use translation model to prepare spoken target texts. In both
cases, we compare numerical entities output from the ITN model.

Table 5 shows ITN entities in bold, the correct spoken and written form
translated entities are in blue and errors are colored with red with correct
form in (parenthesis). When comparing written form English and written form in
target language, we take care of special cases; for example see the following
examples:

  * •

12/24 hour conversion: Use of 24 hours system in French and other target
language;  1:30 p.m. [en] vs. 13h30 [fr].

  * •

Accommodating Zeros: Use alternate magnitudes; e.g., 24,000 [en] vs. 24 mille
[fr].

  * •

Small cardinal: Disambiguate use of spoken/written form for small cardinals
(1-9) based on target language preferences; e.g., two children [en] vs. 2
enfants [fr].

  * •

Separators: Some languages like French use comma(,) for decimal point. Also,
there are languages that use 2 or 3 digit separators with either comma, space
or period; e.g., 25,000.00 [en] vs. 25 000,00 [fr] vs. 25.000,00 [de].

Table 5: Examples of errors for ITN evaluation. The second row is an example of ITN error while the third row is an example of NMT error. Spoken | Written
---|---
|  [en] I found out that nine out of ten
---
statistics are wrong.
|  [en] I found out that 9 out of 10
---
statistics are wrong.
|  [fr] J’ai découvert que neuf statistiques
---
sur dix sont fausses.
|  [fr] J’ai récemment découvert que
---
neuf (9) statistiques sur 10 sont fausses
|  [en] Dad’s surprise sixtieth is on this
---
Saturday. Arrive before six PM.
|  [en] Dad’s surprise 60th is on this
---
Saturday. Arrive before 6 PM.
|  [fr] La soixantième surprise de papa
---
a lieu ce samedi. Arrivée avant 18h
(dix-huit heures).
|  [fr] La 60ème surprise de papa
---
a lieu ce samedi. Arrivée avant 18h.

####  3.2.2 ITN entity translation accuracy

In order to evaluate the NMT model translation accuracy on ITN entities, we
propose the ITN entity translation accuracy. Here we compare whether the
numerical entities from translations produces consistent spoken/written form
output w.r.t. input text form.  An example is provided in the last row of
Table 5. Furthermore, we apply back-translation strategy with the same/other
NMT and compare it with original text ensuring quality dataset from
translation.

###  3.3 Results and Analysis

We show the performance of monolingual baseline models and multilingual models
on the Dictation testset in Table 6. Since the multilingual models have the
same size with the monolingual models, these results are showing the impact of
training data. For the high-resource languages, the performance gain of
multilingual model is quite limited, and even worse performance  is obtained
(e.g., [es]). On the other hand, we observe that the multilingual models
significantly improve the performance on low-resource languages (except Tamil
[ta] since it is using a different script). For example, the normalized
accuracy of monolingual model on Kazakh [kk] is only 0.03%, and the
multilingual models could achieve the performance of 37.69%. Thus, we could
say that adding training data from the same script can improve the model
performance on low-resource languages. As a reference, we also provide the ITN
entities translation accuracy in the last column of Table 6. The higher the
accuracy, the more ITN entities are evaluated in the  Dictation testset.

Similarly, Table 7 shows the  performance of monolingual baseline models  and
the 12-language model on the  Caption Testset. This is computed by  the case A
strategy in Fig. 3. The 12-language ITN model is able to perform better
accuracy on [en, es, de] while maintain competitive accuracy on [fr, it].

From these two tables, we can see that the normalized accuracy is much higher
on the  Dictation testset than on the  Caption Testset.  We find that there is
a domain mismatch in training and testset, as we find a lot of complex math
expressions in the Caption testset (e.g., $2+4+2+7+4=19$) while none is found
in the open subtitles training set.

Table 6: Normalized accuracy of monolingual and multilingual models on the
Dictation testset.

Lang.‡ | Monolingual | 3-lang.‡ | 6-lang.‡ | 12-lang.‡ | ITN trans.‡ acc.‡
---|---|---|---|---|---
es | 79.15% | 78.09% | 76.80% | 75.17% | 91.34%
fr | 62.35% | 62.99% | 60.98% | 60.07% | 62.81%
it | 70.71% | 71.42% | 69.96% | 69.87% | 76.02%
en | 71.73% | - | 72.75% | 71.96% | -
ru | 68.39% | - | 64.66% | 66.33% | 82.86%
kk† | 0.03% | - | 37.69% | 32.41% | 99.63%
tr | 60.07% | - | - | 53.95% | 46.19%
de | 68.24% | - | - | 63.74% | 61.67%
el | 66.84% | - | - | 65.29% | 64.64%
is† | 48.50% | - | - | 61.75% | 99.36%
af† | 29.21% | - | - | 50.51% | 96.45%
ta† | 25.63% | - | - | 27.30% | 99.74%

  * ‡

lang., trans., and acc. stands for languages, translation, and accuracy.

  * †

low resource languages.

Table 7:  Normalized accuracy of monolingual and 12-language  model on the
Caption testset.

Language | en | es | fr | it | de
---|---|---|---|---|---
Monolingual | 63.70% | 64.51% | 55.24% | 57.57% | 48.10%
12-language | 64.74% | 65.58% | 54.90% | 56.77% | 50.19%

Table 8: Normalized accuracy when comparing of architecture and translation tools on 3-langs ([es], [fr], [it]) model and SPM token size of 20,000 on the  Dictation testset. Arch. | NMT | es | fr | it
---|---|---|---|---
Seq2Seq | MetaNMT | 78.09% | 62.99% | 71.42%
Seq2Seq | Opus-MT | 71.11% | 60.03% | 55.89%
Transformer | MetaNMT | 72.55% | 57.27% | 64.76%
Table 9: Comparison of standard rule-based ITN system and data-driven ITN
with/without data augmentation for English.

|  | DD-ITN without | DD-ITN with
---|---|---|---
| Rule-based ITN | data augmentation | data augmentation
en | 63.20% | 51.40% | 86.20%

Finally, Table 8 shows the ITN normalized accuracy when using a different
model architecture and different machine translation model [10]; shows that
the LSTM-based Seq2Seq model is better than the Transformer model, and MetaNMT
is better than the Opus-MT for our task.  Table 9 compares standard rule-based
ITN with DD-ITN and illustrates the impact of utilizing data-driven ITN
modeling with our enhanced text normalization data augmentation for English on
the Google TN evaluation dataset [14] with more than $70K$ ITN entities.

##  4 Conclusion

This work investigates the effectiveness of language agnostic data-driven ITN
model. With the same model size, a 12-languages ITN model can significantly
improves the normalized accuracy of low resource languages while maintain
reasonable performance for high resource languages. Also, we explore the
architecture and the machine translation model used in our framework and found
that Seq2Seq model with MetaNMT is our best system.  Our future work will
include designing better architecture for on-device deployment, evaluation
with clean and native language dataset, adapt data augmentation to support
native language nuances/variation, and expanding to more languages.

## References

  * [1] Ernest Pusateri, Bharat Ram Ambati, Elizabeth Brooks, Ondrej Platek, Donald McAllaster, and Venki Nagesha,  “A mostly data-driven approach to inverse text normalization.,”  in INTERSPEECH. Stockholm, 2017, pp. 2784–2788.
  * [2] Tuan Manh Lai, Yang Zhang, Evelina Bakhturina, Boris Ginsburg, and Heng Ji,  “A unified transformer-based framework for duplex text normalization,”  arXiv preprint arXiv:2108.09889, 2021.
  * [3] Monica Sunkara, Chaitanya Shivade, Sravan Bodapati, and Katrin Kirchhoff,  “Neural inverse text normalization,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 7573–7577.
  * [4] Alexandra Antonova, Evelina Bakhturina, and Boris Ginsburg,  “Thutmose tagger: Single-pass neural model for inverse text normalization,”  arXiv preprint arXiv:2208.00064, 2022.
  * [5] Szu-Jui Chen, Debjyoti Paul, Yutong Pang, Peng Su, and Xuedong Zhang,  “Language Agnostic Data-Driven Inverse Text Normalization,”  arXiv preprint arXiv:2301.08506, 2023.
  * [6] Sepp Hochreiter and Jürgen Schmidhuber,  “Long short-term memory,”  Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  * [7] Minh-Thang Luong, Hieu Pham, and Christopher D Manning,  “Effective approaches to attention-based neural machine translation,”  arXiv preprint arXiv:1508.04025, 2015.
  * [8] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin,  “Attention is all you need,”  Advances in neural information processing systems, vol. 30, 2017\.
  * [9] Taku Kudo and John Richardson,  “Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,”  arXiv preprint arXiv:1808.06226, 2018.
  * [10] Jörg Tiedemann and Santhosh Thottingal,  “OPUS-MT — Building open translation services for the World,”  in Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT), Lisbon, Portugal, 2020.
  * [11] NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, and et al. Heafield,  “No language left behind: Scaling human-centered machine translation,”  2022\.
  * [12] Pierre Lison and Jörg Tiedemann,  “Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles,”  2016\.
  * [13] Nils Reimers and Iryna Gurevych,  “Making monolingual sentence embeddings multilingual using knowledge distillation,”  in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 11 2020, Association for Computational Linguistics.
  * [14] Kaggle Dataset,  “Google Text Normalization Challenge,” https://www.kaggle.com/datasets/google-nlu/text-normalization, 2023,  [Online; accessed 20-Jan-2023].

##  5 Appendix

###  5.1 Data Augmentation with enhanced TN

Unlike a conventional written to spoken text normalization system, our
enhanced text normalization data augmentation system for data-driven ITN is
capable of generating much diversified spoken forms by introducing almost all
possible spoken variations to the written forms as shown in Table 10. Our
rule-based enhance TN system supports cardinal, ordinal, decimal, fraction,
measures, money, date, time and telephone entities. This system performs a
series of steps for each written text as input; pictorially depicted in Figure
1 and describe as follows.

  * a.

Pick: From the text corpus pick up sentences with numerical ITN entities and
discard other sentences.

  * b.

Categorize and Segment: Extract text chunks that matches our carefully
prepared regular expressions for each type of ITN entities in a predefined
order. First, time and date entities segments are collected, followed by
measures, currency, abbreviations. Then we find fractions, decimals, ordinals,
phone numbers and cardinal in respective order.

  * c.

Verbalize: Entities matching text segment are then cleaned, formatted and
normalized with digit and spoken words to its closest verbalized form. For
examples, Time: 12:45 $\rightarrow$ 12 hours 45 minutes.; Date: 12/31/2022 or
(31-12-2022) $\rightarrow$ 31 December 2022; Measures: 10K lb $\rightarrow$
10000 lb.; 207.6 kmps $\rightarrow$ 207.6 kilometers per second.; 2 kg
$\rightarrow$ two kilogram, two kilos, two kilo, etc.

  * d.

Numeric Normalization: With verbalized text, we apply our rewrite mapping
rules with core logic by selecting all possible combination of digits in order
(and apply recursively, if required) to generate $N$ spoken form texts for
corresponding written text.

  * e.

Rewrite: Finally, a rewrite module replaces the written-form text in the
original sentence with $N$ generated diverse spoken forms.

Table 10: Examples of generated spoken form using conventional TN system and our enhanced TN system Input | Conventional TN | Our Augmentation System
---|---|---
123 |  | one hundred
---
twenty three
| one hundred twenty three
---
one twenty three
one hundred and twenty three
one two three
$123 |  | one hundred
---
twenty three dollars
| one hundred twenty three dollars
---
one hundred twenty three dollar
one twenty three dollars
one twenty three dollar
one hundred and twenty three dollars
one twenty three dollars zero cents
123g |  | one hundred
---
twenty three grams
| one hundred twenty three grams
---
one hundred twenty three gram
one twenty three grams
one twenty three gram
one hundred and twenty three grams
one hundred and twenty three gram
one two three grams

###  5.2 Non-ITN Accuracy:

To minimize errors related to entities that are not part of the ITN (i.e.,
non-ITN entities) from our sequence-to-sequence based model, we have
implemented the following measures:

  * a.

Cleansing at source: We filter out spoken-written pairs with non-zero or very
low word-error-rate for non-numerical segments from our training data. It is
important to note that dividing large texts into small segments helps in
achieving highly accurate alignment and thus aids in the removal of errors at
the source.

  * b.

Selective Inference: Recent ASR systems can provide hints about the entity
type of a text segment, which is a simpler task than performing the ITN post-
processing with ASR. Additionally, it is also easy to train a standalone non-
ITN text classifier using a sequence encoder model with our augmentation
dataset. We apply selective inference of numerical entities to ensure that
non-ITN entities are not translated.

  * c.

Accuracy Metric: We have also performed evaluations of non-ITN accuracy for
languages such as Spanish, French, Italian, and German, and found that the
accuracy is greater than 98% on a human-annotated dataset.

###  5.3 Comparison of ITN systems

Here we present experiment and analysis of rule-based ITN system and data-
driven ITN with/without our enhanced text normalization data augmentation
system for English language on evaluation set of Google’s Text Normalization
dataset [14].

The rule-based ITN model, which relies on manually crafted grammars, is
difficult to maintain and prone to errors due to its complexity. Additionally,
it performs poorly on entities that require contextual information. In
contrast, our end-to-end data-driven approach eliminates the need for complex
rules, making it easier to scale to new languages and perform better on
entities requiring contextual information. From the result, it shows that data
driven ITN together with augmentation perform significantly better than rule
based ITN, which achieved 36.4% improvement on overall entity accuracy. We see
significant improvement on Decimal, Measure and Time entities.

Table 11: Comparison of normalized accuracy for rule-based ITN system and data-driven ITN with/without enhanced TN data augmentation for English language. |  | Rule-based | Data-driven ITN | Data-driven ITN
---|---|---|---|---
Class | Size | ITN | without data | with data
|  |  | augmentation | augmentation
Cardinal | 10000 | 92.00% | 55.00% | 97.00%
Ordinal | 10000 | 82.87% | 50.90% | 97.50%
Decimal | 10000 | 2.20% | 53.10% | 90.80%
Measure | 10000 | 46.70% | 42.79% | 88.20%
Telephone | 4024 | 93.20% | 88.00% | 93.80%
Digit | 5442 | 77.60% | 56.80% | 90.60%
Time | 1159 | 36.10% | 35.40% | 73.60%
Date | 10000 | 84.50% | 46.00% | 89.40%
Money | 10000 | 53.80% | 34.90% | 55.40%
Overall | 70625 | 63.20% | 51.40% | 86.20%


Deep learning salt detection

# Complete identification of complex salt-geometries from inaccurate migrated
subsurface offset gathers using Deep Learning

Ana Paula O. Muller11footnotemark: 1 22footnotemark: 2 66footnotemark: 6 Jessé
C. Costa33footnotemark: 3 44footnotemark: 4 Clecio R. Bom11footnotemark: 1
55footnotemark: 5 Elisangela L. Faria11footnotemark: 1 Matheus
Klatt11footnotemark: 1 Gabriel Teixeira11footnotemark: 1 Marcelo P. de
Albuquerque11footnotemark: 1 Marcio P. de Albuquerque11footnotemark: 1

###### Abstract

Delimiting salt inclusions from migrated images is a time-consuming activity
that relies on highly human-curated analysis and is subject to interpretation
errors or limitations of the methods available. We propose to use migrated
images produced from an inaccurate velocity model (with a reasonable
approximation of sediment velocity, but without salt inclusions) to predict
the correct salt inclusions shape using a Convolutional Neural Network (CNN).
Our approach relies on subsurface Common Image Gathers to focus the sediments’
reflections around the zero offset and to spread the energy of salt
reflections over large offsets. Using synthetic data, we trained a U-Net to
use common-offset subsurface images as input channels and the correct salt-
masks as output of a semantic segmentation problem. The network learned to
predict the salt inclusions masks with high accuracy; moreover, it also
performed well when applied to synthetic benchmark data sets that were not
previously introduced. Our training process tuned the U-Net to successfully
learn the shape of complex salt bodies from partially focused subsurface
offset images.

##  1 Introduction

Velocity model building (VMB) is essential to make accurate subsurface images,
especially in regions with high contrast velocities and complex structures.
The conventional model building process uses seismic ray-based tomography
Bishop et al., (1985); Lambare et al., (2014) and Full Waveform Inversion
(FWI) Tarantola, (1984); Pratt, (1999) to determine the velocity model. Each
iteration of Tomography or FWI requires a high level of regularization and a
good input model to avoid local minima or cycle skipping. The result is often
bounded to the maximal resolution power of the method, which often does not
resolve the geological complexity of the area. One good example of the
limitation of conventional inverse methods is the inclusion of salt bodies in
the velocity model, a complex non-automated method that poses a high cost and
is subject to uncertainties.

Salt inclusion is critical during the VMB. Whence a mistake in salt geometry
makes the image below the salt unfocused or distorted Dellinger et al.,
(2017), generating a wrong structure of the subsurface. Therefore, it can lead
to economic consequences, especially in petroleum provinces where the
reservoirs are below complex salt structures. Salt presents a great diversity
of possible geometries Hudec and Jackson, (2007) and has a high velocity
contrast with the enclosing sediments. In complex areas, the definition of the
salt geometry is estimated in an iterative process called salt flood Mosher et
al., (2007). Salt flood iterations require massive interpretation, geological
knowledge of the sedimentary basin, and scenarios testing Herron, (2013);
Jones and Davison, (2014). Moreover, each iteration requires a lot of
computational effort for migrating the data and observing the respective
response of the interpreted salt geometry in the quality of the image
generated. Some works try to overcome such difficulties by focusing on FWI to
correct the badly interpreted salt Michell et al., (2017); Wang et al.,
(2019); Ovcharenko et al., (2018). However, it still requires data acquired
with long offset, wide-azimuth, and low frequency, or the imposition of
geological constraints over the method Esser et al., (2016).

The problem of identifying the presence of salt in seismic images was
previously studied using Deep Learning (DL) Shi et al., (2019); Zhang et al.,
(2019). Nonetheless, this salt-segmentation is made over a well-focused
migrated image. The focalization and accuracy of a seismic image are directly
dependent on an accurate velocity model used in migration, where the model
must include a reasonable estimation of the salt structures above the
interpretation zone. Therefore, the available algorithms for salt-segmentation
do not eliminate the salt flood steps during the VMB. These segmentation
techniques have a high value for interpretation from seismic images and for
refinement of salt structures. However, these structures must be somehow
present in the velocity model used to migrate the seismic data.

Another process that attracts attention in applying DL is velocity estimation
from seismic data Araya et al., (2018); Kazei et al., (2021). Many recent
works are using DL aiming to make the complete prediction of velocity models,
being particularly successful in predicting the salt bodies embedded in such
models, even those with very complex geometries Araya-Polo et al., (2019);
Yang and Ma, (2019); Li et al., (2020). These works use the raw seismic shot
data to train the network to fully predict the correct velocity model that
generated those seismic shots. Their results are very encouraging; however,
they can not be easily extended to current seismic field data. The first
reason is the size of real seismic acquisitions. DL needs to hold the training
data in the device memory, and real seismic shots are much larger than the
most modern device could support, and even if they could be supported, the
number of convolution operations would be prohibitive. Another difficulty is
the irregularity of seismic shots, and the geometric non-correspondence of
this data with the seismic images or the velocity models, i.e., source and
receivers vary their positions during the acquisition process. Concurrent to
our work, Geng et al., (2022), have independently proposed a new approach that
bypasses some problems mentioned here. Their work uses the common image
gathers (CIG) in the angle domain and presents good results in recovering with
DL structurally simple models from synthetic data sets.

Similarly, in this work, we propose to reduce the input data to the imaging
domain to deal with more realistic seismic acquisition geometries, making a
hybrid approach of the conventional VMB workflow and DL velocity estimation
from seismic data. Furthermore, we use DL only in the most challenging step of
VMB flow: the inclusion of salt structures. Instead of using raw shots as
input to the network, we migrated these shots with a reasonable sediment
velocity model, without any salt inclusions. Our migration process uses an
extended imaging condition, to generate a set of images with different
subsurface offsets, which partially focus incorrectly migrated salt
reflections. These images are the input channels for a DL algorithm to solve a
segmentation problem, which has as output the salt mask. Our approach
accelerates the complex workflow for salt estimation, which conventionally
requires multiple migration and interpretation steps. It differs from previous
works on salt segmentation over stacked migrated images Shi et al., (2019);
Alfarhan et al., (2020); Zhang et al., (2019), which fits on an advanced stage
of VMB flow when there is a reasonable estimate for the salt inclusion. The
trained U-Net returned accurate predictions over the test set and also over
benchmark salt models, indicating that this approach has robust features when
compared with traditional methods. Since it requires only focusing
information, it is less sensitive to the velocity model of sediment, i.e., the
training and test sets do not need to present the same model distribution.
Besides, the evaluation of the loss function on the migrated domain attenuates
the effects of coherent and incoherent noise. Moreover, even when the results
present high loss measures, the estimated salt masks are not drastically off,
and can still be used to improve interpretation.

##  2 RTM with extended imaging condition

Reverse Time Migration (RTM) is one of the best algorithms for seismic imaging
data in areas of high complexity Etgen et al., (2009). It can handle high
structural dip and velocity contrast, conditions common in salt basins. The
inputs for RTM are the seismic data shots and the parameters models of the
subsurface. For simplicity and to validate the idea, we will study only
isotropic 2-D models in this work.

RTM uses the full wave equation to back propagate the fields of the receivers
$W_{r}(\mathbf{x},t)$ and to forward propagate the field of the source
$W_{s}(\mathbf{x},t)$ into the earth, where $\mathbf{x}=\\{x,z\\}$ for our 2-D
case. This propagation is made individually for each shot, applying an imaging
condition at each time step of the wave equation solution. In this work, we
will use a cross-correlation extended imaging condition Sava and Vasconcelos,
(2011), whose generalized form can be written as:

$R(\mathbf{x},\mathbf{\lambda},\tau)=\sum_{shots}W_{s}(\mathbf{x}-\mathbf{\lambda},t-\tau)W_{r}(\mathbf{x}+\mathbf{\lambda},t+\tau).$
(1)

Figure 1: Space-lag common-image-gather migrated with (a) correct velocity,
(b) low velocity, and (c) high velocity.

The correlation between source and receiver wave-fields is also evaluated
around the zero lag in the extended imaging condition. Correlations at non-
zero lags indicate that the velocity model is incorrect. Equation 1 opens the
possibility to vary the time and the space lag. For simplicity, we will focus
only on horizontal spatial extension, thus keeping $\tau=0$, and applying
$\mathbf{\lambda}$ only over the $x$ direction. $\lambda$ is known as the
subsurface offset, and it has one interesting property that differs from the
well-known surface offset, which was decisive in choosing this representation
in our study.

Figure 2: Example of one model used to simulate the shots (a) and the
correspondent smoothed version without salt inclusion used for migration (b).
The second line shows the effect of the lack of salt in the migration model
along the common subsurface offset panels ($\lambda$), where the salt
reflections dominate the large offsets due to the velocity error. The
acquisition parameters used to generate this result are described in Section
Modeling Seismic Acquisition.

A gather presenting the subsurface offset, migrated with the correct velocity
model, concentrates the reflections’ energy in $\lambda=0$. The focus of
energy in zero subsurface offset occurs because the source and receiver fields
correlate better at the correct position when using the right model. To
illustrate this property, in Figure 1(a), we show a gather migrated with the
right velocity model. When the velocity used in RTM is not the correct one,
the unfocused energy due to the error in the model spreads through the far
offsets. In Figure 1(b), we can see a gather migrated with low velocity, and
in Figure 1(c), the case with a high velocity. Few studies make use directly
of the subsurface offset gathers Frigério and Biondi, (2021); usually, the
gathers are converted to the angle domain, and these angle gathers continue
the flow of VMB Sava and Fomel, (2003).

In this work, we propose using the property of the spreading energy along
$\lambda$ to detect the high contrast between the velocities of salt and
sediment. Consider that we migrate the data with a reasonable velocity model
of the sediment, without any salt body included, then majorly the salt energy
will spread through the non-zero offset.

In Figure 2 we illustrate an example of inaccurately migrated salt. On the
first line, we show the model used to generate synthetic shots (a) and the
model used in RTM migration (b). The model used in RTM is a smoothed version
of sediment without salt inclusion. On the second line, we show the common
subsurface offset panels, with $\lambda$ = 0 (c), $\lambda$ = 60 m (d) and
$\lambda$ = 120 m (e). As can be seen in Figure 2, the energy of the unfocused
salt will dominate the large offsets. Of course, the shape of salt bodies will
be completely distorted and compressed due to the error in the migration
velocity, and these distortions vary along the $\lambda$. Indeed to infer the
salt geometry using only the inaccurate migrated images presented in Figure 2
is a task which requires mapping the non-linear relations between input and
output data. We will show in the following sections that a CNN can map the
images over the subsurface offsets to the right salt-body shape.

##  3 Data set definition

We generated a set of 1505 synthetics data sets based on marine 2-D models, to
validate the idea of using subsurface offset as input of a CNN to determine
salt geometry. The models have an area of 3 km of depth and 5 km of lateral
extension. The following sections describe the complete process of data set
definition, from choosing the appropriated salt geometries, velocity model
definition, modeling of the shots, and finally migration with the extended
imaging condition.

###  3.1 Salt geometries and the velocity models

In order to get good responses from the network when presented to geological
salt geometries, we obtained the salt geometries used in this work from real
models built with tomography and FWI in the past decade in regions with
complex salt structures and high exploration interest. We extracted salt masks
from these models as 2-D slices with a minimal increment of 0.5 km to achieve
a wide variety of structures. Furthermore, we distorted the proportion between
vertical and horizontal sizes to fit the salt body in the size of our
experiments(3 km x 5 km). With this strategy, we generated a total of 215 salt
masks.

Figure 3: Three examples of models from the set of $1505$ synthetic models
used in this work. Figure (a) includes a geological salt mask, and (b) and (c)
use distortions of this mask.

Besides, we applied distortions in these geological salt masks to increase the
size of the original data set and the variability of geometries presented to
the network. The distortions applied consist of randomly combined operations,
like vertical and horizontal shifts, zoom-in or zoom-out, and rotations.
First, we cut our mask in the top and bottom in 400 m to make the distortions,
working over a 2.2 km of depth mask. These cuts were made to avoid salt
extrusions and ensure that there will be at least one sediment layer below the
bottom of the deepest salt in each model. Then, we included top and bottom
layers of 400 m without salt to return the masks generated to the interest
size.

There were generated six extra masks from the original one, making a total of
1505 salt masks to define the training, validation, and test set. With this
process, we hope to increase our network’s ability to generalize to any salt
geometry. We included each salt mask in different sediment velocity models.
The sediment models are randomly generated obeying some rules. The first layer
has a velocity of 1500 m/s, and the last one has a velocity of 3500 m/s. Each
model has eleven velocity layers with random shapes and depths. The salt body
is then inserted with a velocity of 4500 m/s. In Figure 3 we show three
examples of models from our data set. Figure 3(a) has an original salt mask,
extracted from a real model, Figure 3(b) and Figure 3(c) have a salt obtained
by applying distortions over the original mask.

###  3.2 Modeling seismic acquisition

For each model, we simulated a synthetic seismic acquisition, using a finite-
difference wave propagator, isotropic, acoustic, with second-order in time and
eight-order in space (with spatial sampling equals to 10 m), and CPML
boundaries Martin et al., (2008); Komatitsch and Martin, (2007). To simulate
the source, we used a Ricker wavelet with a peak frequency of 20Hz. The shots
are recorded by 601 receivers with a distance of 10 m between them, in a split
spread acquisition with the maximum offset relative to the source of 3 km.
There were recorded 125 shots, the first shot occurs at the beginning of the
interest region and the last shot at the end, with 40 m of interval between
them. Receivers record at a rate of 4 ms for 3.5 s. We applied the mute of
direct wave in shots for migration. The models were laterally extrapolated to
allow the full shot covering inside the interest area.

We chose these acquisition parameters to ensure a minimum quality level for
the migrated images since the quality of the input images affects the network
performance. However, due to the computational cost of simulations, we did not
try to investigate the optimum parameters for network performance.

###  3.3 Migrated images for training

As mentioned in the previous sections, we migrated our shots with an RTM
cross-correlation extended imaging condition defined in equation 1. The
subsurface offset $\lambda$ starts at 0m , which is equivalent to the non-
extended condition, with an increment of 20 m.

Our main objective in introducing the use of a migration method was to
preserve the CNN’s ability to predict the salt geometries, while reducing the
amount of information to input into the network. We chose RTM due to its
capacity to image complex structures, but other migration methods could be
tested in future extensions of this work.

When we propose to migrate the shots, it is necessary to input also a velocity
model in depth. Thus we gain in reducing the size of inputs, but we lose with
the requirement of such model. Considering that the conventional salt flood
process occurs after the initial definition of the sediment model, it is also
reasonable to include our method in this step of VMB flow. The sediment model
for a salt flood can be a low-frequency model, like the ones obtained with
interactions of tomography and does not need to honor all the details of the
layers. To emulate a reasonable sediment velocity model for salt inclusion, we
smoothed the slowness using a Gaussian filter with $\sigma=12$. Smoothing the
slowness preserves most reflections’ travel time but leaves some errors in the
fine-scale.

##  4 Deep Learning pipeline for salt mask prediction

The presented deep learning pipeline for salt mask prediction is based on an
U-Net architecture. The U-Net Ronneberger et al., (2015) is a type of fully
convolutional network Shelhamer et al., (2016) model widely used in tasks
related to semantic segmentation (e.g., Badrinarayanan et al.,, 2017). The
model’s encoder-decoder structure is ideal for tasks where network inputs and
outputs must have the same spatial dimensions. At least a couple of DL
applications in seismic analysis problems have used the U-Net topology (e.g.,
Alfarhan et al.,, 2020; Wang and Nealon,, 2019), despite not being strictly
related to semantic segmentation tasks. This is possible since these problems
can often be formulated with 2D, or 3D, inputs and outputs of equal spatial
dimensions, which works well with networks of this type.

Here we pose the problem of mapping salt inclusions from migrated sections as
one of binary mask prediction. The U-Net receives a [256, 512, $N_{\lambda}$]
volume as input, where $N_{\lambda}$ is the number of subsurface offsets
panels, and produces as output a [256,512,1] probability map with high peaks
over zero and one, and with the residuals samples concentrated around these
peaks. In this case, the threshold to show the predicted mask is naturally at
the value of 0.5. With respect to the original design Ronneberger et al.,
(2015), we have made only two changes to the architecture: 1) our
convolutional block, in both encoder and decoder sections, contains a batch
normalization layer between each pair of convolutional and activation layers,
which helps to stabilize the gradient flow through the network Ioffe and
Szegedy, (2015); 2) our network starts with 16 feature maps in the first stage
of the encoder, which significantly reduces the number of network parameters
to be trained.

Figure 4: Schematic of our flow. We divided the flow into four steps 1) the
generation of the training data set, 2) migration of shots to reduce the
information and make the data domains compatible for the network, 3) network
training, and finally 4) the salt mask prediction over unseen data.

The four stages encoder section of the network gradually downsamples the data
at each stage and learns to extract relevant information to the task at
different resolution scales of the data. Downsampling occurs only in the first
three encoder stages, where it is performed by max-pooling layers after
processing through the convolutional block. The decoder section gradually
upsamples the feature map back to the original spatial dimensions. It also has
four stages. At each stage the data is first upsampled with a transpose
convolutional layer, then concatenated with the feature map coming from the
skip connection of the corresponding encoding stage, before going through the
convolutional block. After the last upsampling stage in the decoder, a final
convolutional layer is applied to the data to reduce the number of channels to
one, so it matches the single channel of the salt mask labels. Then a sigmoid
activation function is computed over this single panel ([256, 512, 1]), which
represents the network’s predictions.

The total number of trainable parameters is equal to 2,162,225 when using an
input with $N_{\lambda}=7$. We apply the Adam optimizer Kingma and Ba, (2014)
to update the parameters with a learning rate equal to 6e-4. Our model is
built with TensorFlow Abadi et al., (2016). All training processes are carried
out on one NVIDIA V100 with 32Gb, and one training process of 60 epochs takes
approximately ten minutes.

In Figure 4 we summarized our flow from the data generation process to the
predictions over unseen masks. Step 1 is usually adopted in the literature of
VMB with DL, which consists in the generation of synthetic seismic
acquisitions from a known model. This model is used to generate the labels for
the network training.. The main novelties in our flow are in step 2), where we
changed the input data representation to migrated subsurface offsets panels,
and in steps 3) and 4), with training and prediction of the salt mask over
this new input data set. It is essential to clarify that the prediction step
does not require any previous salt geometry information. The RTM common image
gathers reduced the original seismic data size by sixty times, bringing the
network input to a more affordable size. Moreover, the network input and
output are in the same domain and have the same size; such features simplify
the DL approach since the U-Net does not need to perform a domain
transformation.

###  4.1 Training methodology

We trained our network with $N_{\lambda}$ = 7, i.e. with seven subsurface
offsets panels, which varies from $\lambda$ = 0 m to $\lambda$ = 120 m with 20
m of increment. These panels work like channels of the input image for the
CNN. The original size of salt masks and migrated panels is 300 samples in
depth and 500 samples laterally. In order to fit the input and output images
to the proposed U-Net design, we cut the images in depth to 256 and pad 6
samples at each lateral to have images with 256x512 samples. In addition,
since the models have a minimal water layer, we cut the top portion of the
images. We also clipped the amplitude of each subsurface offset panel in
percentile 96 and then normalized the amplitude of each panel between 0 and 1.

We used the $1505$ synthetic models previously described. This data is split
into three distinct sets: training, validation, and test. The training set is
the one that contributes to updating the network’s weights. The validation set
is used to control the metrics and loss during the training process to avoid
overfitting. Finally, the test set is used to make the network predictions
after the training process. We split our original set in twenty percent for
the test set, and for a robust evaluation of network performance, we used
$k$-fold cross-validation in training Moreno-Torres et al., (2012). We
shuffled our validation set in five-folds and then trained the network ten
times for each fold using random initial weights. Our training and validation
loss always shows the statistics over the fifty training processes. Such
strategies account for fluctuations inherent to the random nature of the
process.

One crucial aspect to be considered when training a CNN is to define which
loss function will be used. The loss is the inaccuracy measure of predictions
made by the network and guides the update of the weights through an
optimization algorithm. We used the Jaccard loss, especially because of its
capability to address class imbalances Duque-Arias et al., (2021). Jaccard
loss is expressed as:

$J(y,\hat{y})=1-\frac{\sum_{i}(y_{i}\cdot\hat{y}_{i})}{\sum_{i}y_{i}+\hat{y}_{i}-(y_{i}\cdot\hat{y}_{i})}$
(2)

Figure 5: Figure (a) shows the Jaccard loss and (b) shows the Accuracy
measured along the training process for the training and validation sets.

where $y$ is the correct salt mask and $\hat{y}$ is the predicted one. The
index $i$ refers to the same pixel in the two images. Equation 2 evaluates to
a minimum of 0.0 when $y$ and $\hat{y}$ intersect entirely, and to a maximum
of 1.0 when $y$ and $\hat{y}$ are disjoint. Thus, minimization of the loss
represented by Equation 2 fosters the network model to find solutions that
increase the intersection between $y$ and $\hat{y}$. Such behaviour explains
why the Jaccard loss is ideal for imbalanced problems since it is measured
only over the pixels that compose the objects of interest, in this case, the
true and predicted salt masks. Nevertheless, we also measure the generalist
accuracy score - which is measured for all pixels - as an auxiliary metric.

Figure 5(a) shows the learning curve corresponding to the network performance
and calculated by using the Jaccard loss. The learning curve calculated from
the training data set presents how well the network is learning, while the one
calculated from the validation data set indicates the generalization capacity
of the network. By comparing both curves, it is possible qualitatively infer
that the general behavior of the training curves shows an accurate fit, with a
slight fractional difference between validation loss and training loss. The
validation loss decreases until a point of stability, situated around $50$
epochs, showing a loss median below $0.02$. Figure 5(b) shows the accuracy,
which reaches values up to $0.98$ at the final epochs. This suggests that,
despite of the complexity of the problem addressed, the trained network
generalizes well.

## Results

###  4.2 Images predicted over the original test set

Figure 6: Examples of masks from the test set where the network made
satisfactory predictions, concentrating the errors on the contour details of
the salt inclusions. Blue indicates false positive and red false negative in
error figures, and DC is the Dice Coefficient measure of each example.

Figure 7: Examples of masks from the test set where the network made extensive
mistakes, with predictions that do not include some parts of the salt bodies.
We consider that these errors are associated with regions with low
illumination or the geology is too complex for our simple cross-correlation
image condition. Blue indicates false positive and red false negative in error
figures, and DC is the Dice Coefficient measure of each example.

We evaluated the results over the test set. The test set is composed of models
from the original set that were not seen by the network during training. The
results are presented in Figures 6 and 7, which show the real salt mask, the
predicted salt mask, and the difference between predicted and real masks,
where red indicates false-negative and blue indicates false-positive. To
measure the accuracy of our predictions on each experiment, the Dice
Coefficient Dice, (1945) is printed above each difference panel in Figures 6
and 7. We separated our results into two figures in order to easily discuss
the prediction’s errors observed.

In Figure 6, we captured models with complex salt structures, such as salt
teardrops, overhangs, welds, and tongues, showing that the network can make
surprisingly good predictions even for the more challenging cases. One
astonishing prevision is the correct shape of allochthonous salt, especially
the base of such bodies, and the correct separation from the salt below such
structures. The predictions could capture even thin structures, which suggests
that this approach can also be helpful to track basalt dikes inside the
sediment. The prediction errors made in these masks, even for Figures 6(b) and
6(f), which present the lowest DC, are related to errors in the contour
details of the salt inclusions, and could be easily corrected during the FWI
process or with interpretation over the stacked migrated image with a velocity
model including these predicted salt geometries.

Figure 8: Migrated images using the smooth model without salt inclusions ((a)
and (d)), the smooth model including the predicted salt mask ((b) and (e)),
and the smooth model including the true salt mask ((c) and (f)). The results
on (b) use the predicted mask shown in Figure 7(e), and (e) use the predicted
mask shown in Figure 7(c).

Although, in Figure 7, one can notice examples where the network made
inaccurate predictions, which could jeopardize the migrated image and
compromise the interpretation, with some parts of the salt inclusions that
were not tracked by the network. Below some overhangs, particularly the thick
ones, it is visibly challenging to track the thin salt structures. This is
probably due to the lack of illumination necessary to devise such complex
structures. We can also note border effects in the bottom and laterals,
probably because the energy needed to visualize those structures spread
outside the image domains. Another point of attention is the inaccuracy in
structures containing salt-tongues or diapirs that are upside down due to the
distortion process applied over the geological structures. Our algorithm
usually lacks accuracy in these regions, predicting smaller salt than the
correct one. Such behavior can be explained by the fact that the bottom of a
salt body, when migrated with a smaller velocity, that is our case, collapses
in a smaller region than the actual one. Where the base of salt is a thin
structure, these collapsed regions are probably smaller than the network can
detect as indicative of salt.

In order to clarify the results that could be obtained when using our salt
predictions in a VMB flow, in Figure 8, we generated images by running RTM
using three different models. In the first column, we migrated the synthetic
data with the same models used to generate the images for the
training/prediction process, i.e., a smooth version of the true models without
salt inclusions. Then, in the second column, we included the predicted salt
masks over the models used to generate results in the first column. Finally,
in the third column, we included the true salt masks over the models used to
generate results in the first column. We used the smooth sediment model to
emulate the real VMB process, where the true model could not be entirely
recovered; thus, we are also considering the effect of velocity errors in the
models.

To perform the test presented in Figure 8, we use the models in Figure 7(c)
and (e). They present complex salt structures, like overhangs, whose
delineation requires an iterative process when using a conventional VMB flow.
Our prediction process makes some mistakes in the salt delineation of these
models, as presented in Figure 7(c) and (e). We can see that, despite the
misinterpretations in the salt inclusions, the main structures below the salt
bodies and the overhang are recovered. Besides, the image generated with the
predicted salt can be used to make corrections over the interpreted mask,
improving the image quality. We chose to show the migration without any salt
in Figure 8(a) and (d) only as a reference of the poor quality of images
without salt. It is essential to mention that they correspond to the first
channel of the input images to the U-Net.

###  4.3 Performance evaluation on benchmarks

Figure 9: Figure (a) shows the velocity model for shot simulation including
the salt geometry of a cut in the left portion of the BP/SEG model, and (b)
the smoothed sediment model used in migration to generate the input for
network predictions.
Figure 10: Network prediction for the BP/SEG inspired data. Blue indicates
false positive and red false negative in error figures.

We used two well-known geophysics benchmarks to generate input data sets for
our trained network and evaluate its generalization ability. The benchmark
data sets were not used during the training/validation process. The first one
was a rectangular cutout BP/SEG velocity model Billette and Brandsberg-Dahl,
(2005). We extracted only the salt masks, re-scaled, and inserted them inside
a sediment model as shown in Figure 9(a). The sediment model was defined by
the same algorithm we used to create our original models. Remember that shapes
and velocities of the interfaces are randomly defined but are restricted to a
limited range of possibilities. Therefore, using this data, we want to check
the generalization of salt geometries, not yet the generality for any sediment
velocity. The predicted salt mask for BP/SEG adapted model is shown in Figure
10, where it is possible to see that the network predicted the salt mask with
an accuracy of 0.974, restricting the mistakes to regions where there is
probably a lack of illumination to precisely devise the reflections.

Figure 11: Example of one slice of the SEG 3D model used for shot generation
(a) and the respective model used for migration (b). One remarkable aspect of
these figures is that we do not change sediment’s velocity range, which is
entirely different from the velocity of our training data set.

Another test was performed over 2D velocity models extracted from the 3D SEG
salt model Aminzadeh et al., (1996). In this case, as the SEG model has
dimensions more compatible with the ones we used in our work, we only cut and
reshaped the model to fit in our interest area, leaving the sediment layers
untouched. In Figure 11, we show one slice of the SEG 3D model used in this
work, with the model for shot simulation in Figure 11(a) and the correspondent
model used in RTM in Figure 11(b). The velocity model used in RTM is a
smoothed trace of sediment velocity, which we extended to fulfill our interest
area. For this study, we extracted 24 slices from the original 3D model.

Figure 12: Network predictions for inputs from SEG model. In the error
figures, blue indicates false positive and red false negative in error
figures.

Figure 12 presents four examples of predictions made over the SEG data set,
where it is possible to see that the network generally made accurate
previsions of the salt body location. Here we want to emphasize two
differences from the previous predictions we made. First, sediment velocity
for the shot generation is entirely different from the one used to generate
our training data set. Moreover, the sediment velocity used in migration is
slightly worse than the previous case since we used a plane-parallel model in
migration due to the lack of an isolated sediment model. The previously
mentioned differences in data generated for predictions presented in Figure 12
are a strong indication that our trained network is robust enough to
generalize the predictions to other interest areas.

Another difference between the training set and the SEG test set is that the
salt body velocity is 4482 m/s in the SEG models. Initially, we performed a
velocity manipulation to set the salt velocity to 4500 m/s, which proved
unnecessary with further investigations. The slight deviation from the salt
velocity used in training - 4500 m/s to 4482 m/s - did not change the
predictions significantly; they are almost equal considering both the DC and
the salt body geometry. Extending the tests, we observed that the U-Net
preserves the prediction ability for salt velocities between 4350 m/s and 4900
m/s but loses some accuracy in predictions as the velocity deviates too much
from the training set velocity of 4500 m/s, especially in the contours of salt
inclusions. Despite this encouraging result, we do not believe that pushing
the limits of the trained network is the better approach when predicting salt
geometries in regions where the salt velocity is too different from 4500 m/s.
A better strategy would be to train the U-Net again, using salt models
representing such velocity variation.

###  4.4 Optimal sampling of the subsurface offsets

In the initial input data configuration proposed in this work, we use some
geophysical premises, such as the sampling of subsurface offset. When we build
a velocity model using tomography, it is always desirable to have the offset
information as denser as possible. Such high density avoids alias effects and
makes the assignment of measuring velocity errors easier to be made. In our
problem, we do not have to deal with alias effects because tracking of move-
out information is not done. Nevertheless, if we consider the information
amount, with a large number of offsets, we expect to be giving more
information about the salt velocity errors to the network. Thus there are
important questions to be answered. For example, which is the minimal offset
density necessary for the network to make the correct predictions? Can we
ignore kinematics effects and train the network using only one offset?

To verify these points, we changed the network input and measured the
performance of the loss during the training process. We defined five different
configurations for the input data, as listed below:

  * •

Dense: $N_{\lambda}$ = 7, with $\lambda$ = 0,20,40,60,80,100,120 m

  * •

Sparse: $N_{\lambda}$ = 3, with $\lambda$ = 0,60,120 m

  * •

Near: $N_{\lambda}$ = 1, with $\lambda$ = 0 m

  * •

Mid: $N_{\lambda}$ = 1, with $\lambda$ = 60 m

  * •

Far: $N_{\lambda}$ = 1, with $\lambda$ = 120 m

We evaluated the performance of validation loss using the different inputs,
and observed that the smallest loss is achieved when using the Dense input;
the Sparse input has the second-best performance. This result suggests that
the network uses all the available information to make the best predictions.
When using only one input offset the worst validation loss is achieved, with
almost the same values for Near, Mid and Far offset. Such behavior suggests
that the network accounts for the kinematics effects to make the salt
predictions, being necessary to track the salt reflections positions over the
offsets to detect the salt body shape.

Figure 13: Figure(a) shows the Median Dice Coefficient calculated over each
model in the test set and in the SEG models for the different input
configurations proposed. Figure(b) shows the median Dice coefficient
calculated over the test set when migrated with different sediment velocities.
Smooth are the true velocities smoothed by a Gaussian with $\sigma=12$.
Sediment is the true velocities without salt. Smooth 2 are the true velocities
smoothed by a Gaussian with $\sigma=40$. Mean uses a unique velocity model to
migrate all the models, calculated as the mean of all the velocity models from
our set.

In order to confirm the best way to define the input data, we measured the
Dice Coefficient(DC) for each predicted model, 301 in the test set and 24
models in SEG set. Figure 13(a) shows the median and one standard deviation as
error bars. DC for the test set and SEG models confirms what is observed in
the validation loss curve, presenting the best prediction accuracy when using
the Dense input. Thus, the results degrade when the number of offsets
decreases, reaching the worst results for the Far offset in the test set.
Therefore, Figure 13(a) indicates that our U-Net needs the kinematic
information of salt reflections to make accurate predictions. The worst result
with Far offset, when we use only one, can be explained by the fact that the
top of salt is approximately corrected imaged in the Near offset, due to the
correct sediment velocity, and can guide the interpretation of the top of the
salt body.

The effect of the input subsurface offset is more evident when we analyzed DC
for SEG models. Figure 13(a) shows that the only input configuration that
generates reasonable values of DC is the Dense input. With the others inputs,
the network is not able to generalize to the SEG model. It is also important
to mention here that the maximum subsurface offset of 120 m was chosen based
on the analysis of DC over the Test Set. We tested maximum offset ranging from
80 m to 200 m, and DC reaches the highest value with 120 m, not improving when
using the larger offsets.

###  4.5 How flexible is the trained U-Net?

Another important question is the influence of the velocity model used to
migrate the input data. In Figure 13(b), we make a comparison of the
prediction performance, measuring the Dice Coefficient when migrating the Test
Set using different velocity models. Our original test set is named Smooth,
migrated with the true sediment model smoothed by a Gaussian filter with
$\sigma=12$. Smooth 2 was migrated with a larger Gaussian filter, with
$\sigma=40$. Sediment was migrated with the ground truth sediment velocity
without the salt inclusion, and Mean was migrated using a mean model, which is
calculated as the mean of all sediment velocities.

Figure 13(b) shows that performance depends on the choice of the velocity
models used in migration. The Sediment and Smooth set present almost the same
DC, which varies inside the error bars for Dense, Sparse and Near input
configurations. Thus, with limited velocities errors like the ones present in
the Smooth set, which mainly does not affect the travel time in the migration
algorithm, or with the True sediment model, the U-Net can recover the salt
structures correctly. However, when using velocity models with larger errors,
like those in Smooth 2 and Mean sets, the DC reduces, indicating a
misinterpretation of the salt. Consequently, Figure 13(b) confirms that the
conventional practice of inserting the salt body only when the sediment
velocity is well resolved is also important for our method.

Figure 14: Figures (a-c) shows the predictions when using the Sparse, Near and
Mid configurations respectively, made for the salt mask from SEG model in
Figure 12(b). Figures (d-f) shows the predictions made with the Dense
configuration, but using the Sediment, Smooth2 and Mean models to migrate the
input data for the salt mask in Figure 6(a). Blue indicates false positive and
red false negative in error figures.

In Figure 14 we exemplified the effects observed statistically in Figure 13.
Figures 14(a-c) show the predictions made over one SEG model (Figure 12(b)),
when using a limited number of input subsurface offsets. It is possible to
observe that the predicted images preserve the salt body, but also incorrectly
classify sediment areas as salt, confirming that the complete numbers of
offsets are necessary to segment the salt from the sediment appropriately. In
Figures 14(d-f), we show the effect of using different migrations velocities
in a Dense configuration, for the model in Figure 6(a). Checking the predicted
images, we observed that with the True sediment model Figure 14(a), the salt
structures were predicted with almost the same quality observed with the
Smooth model (Figure 6(a)). When increasing the error in the model,
represented by Figure 14(b-c), the shape of the salt is almost entirely
recovered but presents vertical displacements and lateral misinterpretations.
These features can be explained by the fact that errors in the sediment
velocities change the position of structures in migrated images. Thus, the
U-Net still preserves the ability to identify the salt, but since the sediment
velocity model is incorrect, the imaged salt structure presents distortions.

##  5 Discussion

This work proposed a novel use for the subsurface offset, which considers the
property that incorrectly migrated reflection spreads through the large
subsurface offsets. We trained a U-Net to predict in one step the complete
salt inclusion, using as input the shots migrated with a smooth sediment
velocity. Migrating the shots has two evident improvements over the known
strategies of using the shots directly: reduces/regularizes information and
brings the network input to the same domain of network output. As we converted
the input data to the domain of the output data, this problem can be extended
in size or dimensions. However, the migration strategy imposes some
requirements not present when using the shots directly, such as the
computational cost of migrating the data and the necessity of a well-resolved
sediment velocity. Such requirements are new for the VMB flow using DL.
However, they are still present in conventional salt inclusion flow, which
occurs in a stage when the sediment velocity is well-resolved and requires
many migrations until the iterative process of salt inclusion is completed
Dellinger et al., (2017).

The trained U-Net can partially recover the salt structures using different
degrees of smoothness for the sediment velocity, which indicates that the
predictions are robust even when we do not know the exact sediment velocity.
Besides, migration introduced a generalization ability in the network, as this
eliminates the effect of the background velocity. We show with SEG models that
we can use a trained network to predict salt even when the sediment velocity
is completely different from the one used in the training process, as long as
the salt velocity is a value close to the one used for training and the salt
geometry in the training set is sufficiently diverse. Such generalization is
relevant because the most expensive step is generating synthetic data to train
the network. Once we have the trained network, predictions are made almost
instantly.

When applying an imaging method to reduce our input information and focus only
on the desirable feature (salt geometry), we introduced some well-known
imaging problems in the predictions, such as illumination, band-limited
resolution, and border effects. We chose our survey design, sample interval
and wavelet parameters based on previous experience with imaging methods,
reflecting a balanced choice between image quality and computational cost.
Since the quality of predictions are directly related to the quality of the
images we input in the U-Net, these questions can be addressed in future
investigations of our approach. For example, using seismic data with wide
frequency-band, image condition varying the $\lambda$ horizontally and
vertically, time-shift extended imaging condition, and processing improvements
to CIGs. Moreover, we believe that an extension to 3D models can better handle
the illumination problems that we have in some predictions because the
structures could be illuminated in different directions.

Applying the salt inclusion technique proposed in this work to real seismic
data requires additional studies to deal with some questions. Since real
acquisitions in complex areas are usually 3D data, one question is the choice
of an appropriate CNN architecture to train with 3D images Çiçek et al.,
(2016); Wu et al., (2019); Chen et al., (2016); Lee et al., (2021) and that
can capture the accurate salt inclusions. Since we are training with synthetic
data, another question is how to include effects related to noise, anisotropy
and multiples in simulations. Another point of attention is that field data
could have salt inclusions more complex than the ones we used here, with
stratified salt and salt bodies surrounded by complex sediment or high-
velocity sediment in deeper areas. Probably the mentioned points should be
considered when constructing a training set. Besides, the phase and amplitude
spectrum of the source wavelet in field data vary from the one used in the
training process. Thus, it should be necessary to make these wavelets similar
before the migration by applying a matching filter Kazei et al., (2020).

##  6 Conclusions

This work presented an alternative approach to the conventional salt flood
methodology, which consists of training a DL network (U-Net) to locate salt
inclusions over incorrectly migrated subsurface offset panels. The velocity
model used in migration must be a good representation of sediment velocities
but without any salt inclusion. In migrated images, salt events are entirely
distorted from the original geometry due to the lack of salt velocity in the
migration model. We showed that a U-Net can successfully learn to identify and
make the non-linear transforms necessary to estimate the salt location when we
input a sufficient number of subsurface offsets in the U-Net and a reasonable
sediment velocity in migration algorithm. Despite minor inaccuracies, our
predicted salt bodies highly correlate with the actual structures. Moreover,
when they are included in the migration model, the identification of salt
geometry is improved. Thus, even if they can not be used as a final salt
interpretation, they can be an excellent initial approximation of the true
salt geometry, either for improving the salt body interpretation without the
extensive testing of salt scenarios or as a better approximation of salt
inclusions for FWI iterations.

##  7 Acknowledgements

APOM thanks Petrobras for sponsoring her postdoctoral research and for the
permission to publish this work. JCC acknowleges the CNPq financial support
through the INCT-GP and the grant 312078/2018-8 and Petrobras. CRB
acknowledges the financial support from CNPq (316072/2021-4) and from FAPERJ
(grants 201.456/2022 and 210.330/2022). The authors acknowledge the
LITCOMP/COTEC/CBPF multi-GPU development team for all the support in the
Artificial Intelligence infrastructure and Sci-Mind’s High-Performance multi-
GPU system. The authors also would like to thank the SENAI CIMATEC
Supercomputing Center for Industrial Innovation, for the cooperation, supply
and operation of computing facilities.

## References

  * Abadi et al., (2016) Abadi, M., A. Agarwal, P. Barham, E. Brevdo, and Z. Chen, 2016, Tensorflow: Large-scale machine learning on heterogeneous distributed systems: arXiv, abs/1603.04467.
  * Alfarhan et al., (2020) Alfarhan, M., A. Maalej, and M. Deriche, 2020, Concurrent detection of salt domes and faults using resnet with u-net: 2020 6th Conference on Data Science and Machine Learning Applications (CDMA), 118–122.
  * Aminzadeh et al., (1996) Aminzadeh, F., N. Burkhard, J. Long, T. Kunz, and P. Duclos, 1996, Three dimensional seg/eaeg models — an update: The Leading Edge, 15, 131–134.
  * Araya et al., (2018) Araya, M., J. Jennings, A. Adler, and T. Dahlke, 2018, Deep-learning tomography: The Leading Edge, 37, 58–66.
  * Araya-Polo et al., (2019) Araya-Polo, M., S. Farris, and M. Florez, 2019, Deep learning-driven velocity model building workflow: The Leading Edge, 38, 872a1–872a9.
  * Badrinarayanan et al., (2017) Badrinarayanan, V., A. Kendall, and R. Cipolla, 2017, Segnet: A deep convolutional encoder-decoder architecture for image segmentation: IEEE Transactions on Neural Networks and Learning Systems, 39, 2481 – 2495.
  * Billette and Brandsberg-Dahl, (2005) Billette, F., and S. Brandsberg-Dahl, 2005, The 2004 bp velocity benchmark: Presented at the 67th EAGE Conference & Exhibition, European Association of Geoscientists & Engineers.
  * Bishop et al., (1985) Bishop, T. N., K. P. Bube, R. T. Cutler, R. T. Langan, P. L. Love, J. R. Resnick, R. T. Shuey, D. A. Spindler, and H. W. Wyld, 1985, Tomographic determination of velocity and depth in laterally varying media: Geophysics, 50, 903–923.
  * Chen et al., (2016) Chen, J., L. Yang, Y. Zhang, M. Alber, and D. Z. Chen, 2016, Combining fully convolutional and recurrent neural networks for 3d biomedical image segmentation: Advances in Neural Information Processing Systems (NIPS), 29, 3036–3044.
  * Çiçek et al., (2016) Çiçek, Ö., A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, 2016, 3d u-net: Learning dense volumetric segmentation from sparse annotation: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016, Springer International Publishing, 424–432.
  * Dellinger et al., (2017) Dellinger, J., A. J. Brenders, J. R. Sandschaper, C. Regone, J. Etgen, I. Ahmed, and K. J. Lee, 2017, The garden banks model experience: The Leading Edge, 36, 151–158.
  * Dice, (1945) Dice, L. R., 1945, Measures of the amount of ecologic association between species: Ecology, 26, 297–302.
  * Duque-Arias et al., (2021) Duque-Arias, D., S. Velasco-Forero, J.-E. Deschaud, F. Goulette, A. Serna, E. Decencière, and B. Marcotegui, 2021, On power jaccard losses for semantic segmentation: Presented at the VISAPP 2021: 16th International Conference on Computer Vision Theory and Applications.
  * Esser et al., (2016) Esser, E., L. Guasch, F. Herrmann, and M. Warner, 2016, Constrained waveform inversion for automatic salt flooding: The Leading Edge, 35, 235–239.
  * Etgen et al., (2009) Etgen, J., S. H. Gray, and Y. Zhang, 2009, An overview of depth imaging in exploration geophysics: Geophysics, 74, no. 6, WCA5–WCA17.
  * Frigério and Biondi, (2021) Frigério, O., J., and B. Biondi, 2021, Tomographic waveform inversion (twi): Presented at the IMAGE - International Meeting for Applied Geoscience & Energy, SEG —AAPG.
  * Geng et al., (2022) Geng, Z., Z. Zhao, Y. Shi, X. Wu, S. Fomel, and M. Sen, 2022, Deep learning for velocity model building with common-image gather volumes: Geophysical Journal International, 228, 1054–1070.
  * Herron, (2013) Herron, D. A., 2013, Thoughts and observations on interpreting depth-imaged data: Interpretation, 1, B1–B6.
  * Hudec and Jackson, (2007) Hudec, M. R., and M. P. Jackson, 2007, Terra infirma: Understanding salt tectonics: Earth-Science Reviews, 82, 1–28.
  * Ioffe and Szegedy, (2015) Ioffe, S., and C. Szegedy, 2015, Batch normalization: Accelerating deep network training by reducing internal covariate shift: ArXiv, abs/1502.03167.
  * Jones and Davison, (2014) Jones, I. F., and I. Davison, 2014, Seismic imaging in and around salt bodies: Problems and pitfalls: SEG Technical Program Expanded Abstracts, 3684–3688.
  * Kazei et al., (2020) Kazei, V., O. Ovcharenko, and T. Alkhalifah, 2020, Velocity model building by deep learning: From general synthetics to field data application: SEG Technical Program Expanded Abstracts, 1561–1565.
  * Kazei et al., (2021) Kazei, V., O. Ovcharenko, P. Plotnitskii, D. Peter, X. Zhang, and T. Alkhalifah, 2021, Mapping full seismic waveforms to vertical velocity profiles by deep learning: Geophysics, 86, no. 5, R711–R721.
  * Kingma and Ba, (2014) Kingma, D. P., and J. Ba, 2014, Adam: A method for stochastic optimization: arXiv, abs/1412.6980.
  * Komatitsch and Martin, (2007) Komatitsch, D., and R. Martin, 2007, An unsplit convolutional perfectly matched layer improved at grazing incidence for the seismic wave equation: Geophysics, 72, no. 5, SM155–SM167.
  * Lambare et al., (2014) Lambare, G., P. Guillaume, and J. Montel, 2014, Recent advances in ray-based tomography: European Association of Geoscientists & Engineers, 2014, 1–5.
  * Lee et al., (2021) Lee, K., L. Sunwoo, T. Kim, and K. J. Lee, 2021, Spider u-net: Incorporating inter-slice connectivity using lstm for 3d blood vessel segmentation: Applied Sciences, 11.
  * Li et al., (2020) Li, S., B. Liu, Y. Ren, Y. Chen, S. Yang, Y. Wang, and P. Jiang, 2020, Deep-learning inversion of seismic data: IEEE Transactions on Geoscience and Remote Sensing, 58, 2135–2149.
  * Martin et al., (2008) Martin, R., D. Komatitsch, and A. Ezziani, 2008, An unsplit convolutional perfectly matched layer improved at grazing incidence for seismic wave propagation in poroelastic media: Geophysics, 73, no. 4, T51–T61.
  * Michell et al., (2017) Michell, S., X. Shen, A. Brenders, J. Dellinger, I. Ahmed, and K. Fu, 2017, Automatic velocity model building with complex salt: Can computers finally do an interpreter’s job?: SEG Technical Program Expanded Abstracts, 5250–5254.
  * Moreno-Torres et al., (2012) Moreno-Torres, J. G., J. A. Sáez, and F. Herrera, 2012, Study on the impact of partition-induced dataset shift on $k$-fold cross-validation: IEEE Transactions on Neural Networks and Learning Systems, 23, 1304–1312.
  * Mosher et al., (2007) Mosher, C., E. Keskula, J. Malloy, R. Keys, H. Zhang, and S. Jin, 2007, Iterative imaging for subsalt interpretation and model building: The Leading Edge, 26, 1361–1488.
  * Ovcharenko et al., (2018) Ovcharenko, O., V. Kazei, D. Peter, and T. Alkhalifah, 2018, Variance-based model interpolation for improved full-waveform inversion in the presence of salt bodies: Geophysics, 83, no. 5, R541–R551.
  * Pratt, (1999) Pratt, R. G., 1999, Seismic waveform inversion in the frequency domain — part 1: Theory and verification in a physical scale model: Geophysics, 64, 888–901.
  * Ronneberger et al., (2015) Ronneberger, O., P. Fischer, and T. Brox, 2015, U-net: Convolutional networks for biomedical image segmentation: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 234–241.
  * Sava and Vasconcelos, (2011) Sava, P., and I. Vasconcelos, 2011, Extended imaging conditions for wave-equation migration: Geophysical Prospecting, 59, 35–55.
  * Sava and Fomel, (2003) Sava, P. C., and S. Fomel, 2003, Angle‐domain common‐image gathers by wavefield continuation methods: Geophysics, 68, 1065–1074.
  * Shelhamer et al., (2016) Shelhamer, E., J. Long, and T. Darrell, 2016, Fully convolutional networks for semantic segmentation: IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 1–1.
  * Shi et al., (2019) Shi, Y., X. Wu, and S. Fomel, 2019, Saltseg: Automatic 3d salt segmentation using a deep convolutional neural network: Interpretation, 7, 1A–T725.
  * Tarantola, (1984) Tarantola, A., 1984, Inversion of seismic reflection data in the acoustic approximation: Geophysics, 49, 1259–1266.
  * Wang and Nealon, (2019) Wang, E., and J. Nealon, 2019, Applying machine learning to 3d seismic image denoising and enhancement: Interpretation, 7, 1–34.
  * Wang et al., (2019) Wang, P., Z. Zhang, J. Mei, F. Lin, and R. Huang, 2019, Full-waveform inversion for salt: A coming of age: The Leading Edge, 38, 204–213.
  * Wu et al., (2019) Wu, X., L. Liang, Y. Shi, and S. Fomel, 2019, Faultseg3d: Using synthetic data sets to train an end-to-end convolutional neural network for 3d seismic fault segmentation: Geophysics, 84, no. 3, IM35–IM45.
  * Yang and Ma, (2019) Yang, F., and J. Ma, 2019, Deep-learning inversion: A next-generation seismic velocity model building method: Geophysics, 84, no. 4, R583–R599.
  * Zhang et al., (2019) Zhang, L., M. Zhang, Z. Zhong, T. Zhao, Y. Wu, J. Wei, and C. Zhan, 2019, Deep learning approach in characterizing salt body on seismic images: SEG Technical Program Expanded Abstracts, 2594–2598.