# Reinforcement Learning for Optimizing RAG for Domain Chatbots

Mandar Kulkarni, Praveen Tangarajan, Kyung Kim, Anusua Trivedi

###### Abstract

With the advent of Large Language Models (LLM), conversational assistants have
become prevalent for domain use cases. LLMs acquire the ability to contextual
question answering through extensive training, and Retrieval Augmented
Generation (RAG) further enables the bot to answer domain-specific questions.
This paper describes a RAG-based approach for building a chatbot that answers
user’s queries using Frequently Asked Questions (FAQ) data. We train an in-
house retrieval embedding model using infoNCE loss, and experimental results
demonstrate that the in-house model works significantly better than the well-
known general-purpose public embedding model, both in terms of retrieval
accuracy and Out-of-Domain (OOD) query detection. As an LLM, we use an open
API-based paid ChatGPT (gpt-35-turbo-16k-0613) model. We noticed that a
previously retrieved-context could be used to generate an answer for specific
patterns/sequences of queries (e.g., follow-up queries). Hence, there is a
scope to optimize the number of LLM tokens and cost. Assuming a fixed
retrieval model and an LLM, we optimize the number of LLM tokens using
Reinforcement Learning (RL). Specifically, we propose a policy-based model
external to the RAG, which interacts with the RAG pipeline through policy
actions and updates the policy to optimize the cost. The policy model can
perform two actions: to fetch FAQ context or skip retrieval. We use the open
API-based GPT-4 as the evaluation model, which rates the quality of the bot
answer. Using an appropriate reward shaping, GPT-4 ratings are converted to a
numeric reward. We then train a policy model using policy gradient on multiple
training chat sessions. As a policy model, we experimented with a public gpt-2
model and an in-house BERT model. With the proposed RL-based optimization
combined with similarity threshold, we are able to achieve significant (~31%)
cost savings while getting a slightly improved accuracy. Though we demonstrate
results for the FAQ chatbot, the proposed RL approach is generic and can be
experimented with any existing RAG pipeline.

## Introduction

With the advent of Large Language Models (LLM), we observe an increased use of
conversational assistants even for the domain use cases. Trained on a large
web-scale text corpus with approaches such as instruction tuning and
Reinforcement Learning with Human Feedback (RLHF), LLMs have become good at
contextual question-answering tasks, i.e., given a relevant text as a context,
LLMs can generate answers to questions using that information. Retrieval
Augmented Generation (RAG) is one of the key techniques used to build chatbots
for answering questions on domain data. RAG consists of two components: a
retrieval model and an answer generation model based on LLM. The retrieval
model fetches context relevant to the user’s query. The query and the
retrieved context are then inputted to the LLM with the appropriate prompt to
generate the answer. Though intuitive to understand, RAG-based chatbots have a
few challenges from a practical standpoint.

  1. 1.

If a retrieval model fails to fetch relevant context, the generated answer
would be incorrect and uninformative. The retrieval is more challenging for
non-English queries, e.g., code-mix Hinglish.

  2. 2.

For paid API-based LLMs (e.g., ChatGPT), the cost per call is calculated based
on the number of input and output tokens. A large number of tokens passed in a
context leads to a higher cost per API call. With a high volume of user
queries, the cost can become significant.

  3. 3.

To enable multi-turn conversations with RAG, a conversation history needs to
be maintained and passed to the LLM with every query. It is known that a
larger input token size leads to a drop in accuracy or hallucinations as LLMs
have an additional task of choosing relevant information from a large context
(Liu et al. 2023a).

In this paper, we first describe a RAG-based approach for building a chatbot
that answers user’s queries using Frequently Asked Questions (FAQ) data. We
have a domain FAQ dataset consisting of 72 FAQs regarding the credit card
application process. The FAQ dataset is prepared to answer user queries
regarding general card information pre- and post-application queries. We train
an in-house retrieval (embedding) model using info Noise Contrastive
Estimation (infoNCE) loss (van den Oord, Li, and Vinyals 2019) with the
English and Hinglish paraphrase queries created using ChatGPT. The embedding
model is trained to maximize query-question and query-QnA similarity.
Experimental results show that the in-house model performs significantly
better than a public pre-trained embedding model regarding retrieval accuracy
and Out-of-Domain (OOD) query detection. OOD queries are questions unrelated
to the domain data, e.g., how is the weather today? For every user query, we
retrieve top-k FAQs (question + answer) as context and input them to the LLM
to generate the answer. We maintain the previous two queries, answers, and FAQ
context as history to enable the multi-turn conversation. We use an open paid
API-based ChatGPT (gpt-35-turbo-16k-0613) for all our experiments as an LLM.
GPT-4 is used to rate the bot answer quality (as Good/Bad) for the end-to-end
RAG pipeline.

We propose an RL-based approach to optimize the number of tokens passed to the
LLM. We noticed that for certain patterns/sequences of queries, we can get a
good answer from the bot even without fetching the FAQ context. Examples of
such scenarios can be: 1. for a follow-up query; FAQ context need not be
retrieved if it has already been fetched for the previous query; 2. for the
sequence of queries referring to the same FAQ, a context can be fetched only
once at the start; 3. for OOD queries, the LLM prompt itself can guide the bot
to generate the answer. Using this insight, we propose a policy gradient-based
approach to optimize the number of LLM tokens and, hence, the cost. The input
to the policy model is the State, which comprises previous queries, previous
policy actions, and the current query. The policy model can take two actions:
[FETCH] or [NO_FETCH]. A usual RAG pipeline would be executed when a policy
network takes a [FETCH] action. When a policy network chooses a [NO_FETCH]
action, FAQ context is not retrieved. A query and context (empty in the case
of [NO_FETCH] action) are inputted to the LLM. We use GPT-4 as the reward
model and convert the quality rating (Good/Bad) to the numeric reward using
appropriate reward shaping. If the LLM generates a good answer (as rated by
GPT-4) even without fetching the context, we give it a high positive reward to
promote such actions in the future. If not fetching a FAQ context leads to a
wrong answer, we provide a negative reward (e.g., this can happen if the
policy model chooses a [NO_FETCH] action for a domain query without any
previous relevant context). For the training chat sessions, for each State, a
policy model samples an action according to the current probability
distribution over the actions. We then generate (State, Action, Reward)
trajectories by sampling multiple times from the current policy. A policy
model is then updated using a policy gradient with cumulative reward. As a
policy model, we experimented with in-house BERT and public gpt-2 models.
Figure 1 shows the architecture of the proposed policy-based approach.
Experimental results demonstrate that the policy model provides token saving
by fetching the FAQ context only when it is required. When combined with a
simple similarity threshold-based optimization, we are able to achieve token
savings of ~31% on the test chat session with 91 queries while achieving
slightly improved accuracy (evaluated through manual labeling) than the usual
RAG pipeline. This underlines the effectiveness of the proposed approach.

## Related works

With recent advancements in Generative AI and LLMs, the Retrieval-Augmented
Generation (RAG) (Lewis et al. 2021) approach has emerged as the preferred
strategy for contextual question answering. RAG pipeline consists of a
retrieval model followed by an LLM to generate the answer. Different
approaches have been experimented with to improve components of the RAG in
terms of accuracy and minimize hallucinations in answer generation. Khatry et
al. (kha 2023) proposed a low-rank residual adaptation approach with the pre-
trained embedding model to improve the retrieval model. It was shown to lead
to improved task-specific retrieval as compared to a general-purpose
embeddings-based baseline. Instead of using an interleaved retrieval and
generation, Shao et al. (Shao et al. 2023) proposed an iterative retrieval and
generation approach where the current model output acts as an informative
context for retrieving more relevant knowledge which in turn helps generate a
better output in the next iteration. Li et al. (Li et al. 2022) extensively
surveys recent RAG-based approaches.

RL has been experimented with to improve RAG. Bacciu et al. (Bacciu et al.
2023) propose an RL-based approach to train an efficient retriever model to
search for relevant information in an arbitrarily large database. Once this
set of relevant data has been retrieved, it is forwarded to the API-based LLM
to generate the answer. In particular, the authors show that RL helps reduce
hallucinations by minimizing the number of damaging documents returned by the
retriever. Self-RAG (Asai et al. 2023) trains a single LLM that adaptively
retrieves passages and generates and reflects on retrieved passages and their
generations using reflection tokens. In our work, we assume we do not have
access to the gradients of the retrieval model and LLM. We only train a policy
model that resides external to RAG.

GPT-4 is observed to provide human-level accuracy for automated evaluation
tasks. Hack at al. (Hackl et al. 2023) investigated the consistency of
feedback ratings generated by GPT-4 across multiple iterations, time spans,
and stylistic variations. The study indicated that GPT-4 can generate
consistent ratings across repetitions with a clear prompt. Liu et al. (Liu et
al. 2023b) evaluated GPT-4 with two generation tasks: text summarization and
dialogue generation. It was shown that GPT-4 provided state-of-the-art results
on automated evaluation, which highly correlates with human evaluation. In our
work, we use GPT-4 for automated evaluation of bot responses without ground
truth answers.

## RAG for FAQ chatbots

In this section, we describe details of the datasets and embedding model
training for RAG for FAQ chatbots.

### Dataset

We have a FAQ dataset consisting of 72 FAQs regarding the credit card
application process. The FAQ dataset is prepared to answer user queries
regarding general card information (e.g., annual fee, benefits), pre-
application issues faced on the platform (e.g., server issue, content not
available, not getting One Time Password (OTP)), and post-application services
(e.g., increasing credit limit, purchase EMI options). The FAQs list is
designed based on the user survey conducted on the platform, collecting
information about current pain points faced by users about the application.

### Training in-house embedding model

To use RAG for FAQ chatbots, we first need to retrieve the top k most relevant
FAQs to answer the given query. To fetch top-k FAQs, an embedding model is
used to encode queries and FAQs. We concatenate a question and an answer
within a FAQ and use this to get a vector for that FAQ. A cosine similarity
measure is then used to rank the FAQs based on their relevance. We
experimented with popular general-purpose public pre-trained embedding models
and models trained on in-domain data. We trained an in-house embedding model
with infoNCE loss (van den Oord, Li, and Vinyals 2019) using a well-known
embedding model (e5-base-v2) weights as initialization. InfoNCE has proved an
effective loss for contrastive learning, specifically under self-supervised
setting (Goyal et al. 2021). We finetune a publicly available embedding model
e5-base-v2 (Wang et al. 2022) using in-domain data with infoNCE loss.

We create a dataset for training the embedding using ChatGPT and small manual
tagging. We generate multiple English queries per FAQ as question paraphrases.
Due to the large non-English-speaking population in India, we observe many
Hinglish queries on the platform. We also generate a few Hinglish paraphrase
queries per FAQ to support code-mix Hinglish queries. Next, we add a small set
of manually tagged corpus where we add queries that can be answered based on
the FAQ answers’ content. We use ~3.5k queries for training, ~1k queries for
validation, and 1014 queries for testing.

We use the following loss function to train the model.

$\displaystyle
l_{i,j}=\frac{\exp(\text{sim}(z_{i},z_{j})/\tau)}{\sum_{k=1}^{2B}\mathds{1}_{[k\neq
i]}\exp(\text{sim}(z_{i},z_{k})/\tau)}$ (1)

Here, $z_{i}$ and $z_{j}$ indicate a positive pair (i.e., question paraphrases
for the same FAQ), $B$ indicates batch size, and $\tau$ indicates temperature.
For all our experiments, we set $N$ to 8 and $\tau$ to 0.1 and use cosine
similarity for sim.

As seen from Eq. 1, infoNCE loss is specifically suitable for training with
query-FAQ mapped data because it only needs positive pairs and treats
remaining samples as in-batch negatives. We finetune the embedding model with
two objectives: maximize query-QnA similarity and query-question similarity.

Table 1 shows the top-1 and top-3 accuracy comparison results for English and
Hinglish queries with a general-purpose public model (e5-base-v2), the models
finetuned with triplet loss and infoNCE loss. The ranking results are computed
based on query-QnA cosine similarity. It can be seen that the in-house model
trained with infoNCE works significantly better than the public pre-trained
model.

Model | English | Hinglish
---|---|---
| top-1 | top-3 | top-1 | top-3
e5-base-v2 | 0.82 | 0.91 | 0.71 | 0.87
triplet-loss | 0.90 | 0.93 | 0.84 | 0.89
infoNCE | 0.97 | 0.98 | 0.94 | 0.95
Table 1: Retrieval accuracy comparison of public vs in-house embedding models

Next, we compare the detection performance of public and in-house models on
in-domain vs out-of-domain (OOD) samples. We created a dataset of 30 in-domain
and OOD queries each. In-domain queries contain English and Hinglish queries,
while OOD queries include greeting, acknowledgment, and general non-domain-
related questions. We calculated mean scores for positive and negative queries
with the public and finetuned model. Table 2 shows the average of top-1
matching scores for the two models. Even though the in-house model is not
specifically trained on OOD queries, it performs much better in OOD query
identification. A general-purpose pre-trained model assigns less
discrimination scores for in-domain and OOD queries. Therefore, with a public
embedding model, it’s not possible to use a similarity threshold. As shown
later, such a threshold can help token optimization for FAQ chatbots.

Model | In-domain | OOD
---|---|---
e5-base-v2 | 0.84 | 0.82
finetuned with InfoNCE | 0.85 | 0.56
Table 2: Comparison of top-1 cosine similarity scores for in-domain vs OOD
queries
---
Figure 1: Proposed policy agent based architecture for optimizing RAG for
domain chatbots.

### Prompt for ChatGPT

Since this is a finance-related bot, we attempted to design a prompt that
minimizes hallucinations with ChatGPT. Apart from adding a relevant prompt for
contextual QnA, we add in-context examples to get the results in the required
JSON format. Adding a post prompt was observed to help with reducing
hallucinations. A post prompt is a small text instruction passed to ChatGPT
along with every query. In our case, the post prompt is ’Instruction: Answer
only using the information available.’. To avoid providing irrelevant
information for OOD queries, ChatGPT is prompted to provide a "do not know"
answer.

For all our experiments, we set k to 3. For a ChatGPT call, we maintain the
last two conversations as a history. The conversation history includes
previous queries, generated bot answers, and faq context. We need to maintain
FAQ context in conversation history to answer follow-up questions. E.g. for
the sequence of two queries [’is there annual fee’, ’can you reduce it?’], a
FAQ context fetched for the second query would be irrelevant, and ChatGPT
needs to rely on the FAQ context fetched for the first query.

### Evaluation

With the lack of ground truth answers with a generative model such as ChatGPT,
off-the-shelf evaluation metrics such as BLEU or ROUGE cannot be used. To
evaluate the quality of answers generated from the bot, we use the GPT-4-based
approach. GPT-4 has shown promising results for automated quality evaluation
compared to human evaluation (Hackl et al. 2023) (Liu et al. 2023b). We
provide GPT-4 with the current query, current FAQ context, answer generated
from the bot, and conversation history (which includes previous query,
answers, and FAQ context). We do not provide GPT-4 with the ideal or ground
truth response. Instead, we prompt GPT-4 to evaluate the quality of the
generated answer based on the information available in the current FAQ context
and conversation history. GPT-4 is instructed to evaluate the output based on
whether it exactly matches the information provided in the FAQ context. Any
additions or deviations from the available information are termed as factual
failures and are to be rated bad. GPT-4 is asked to provide a final rating as
good or bad and the reason behind the rating. For the OOD queries, the bot is
supposed to provide a "do not know" answer, and GPT-4 is instructed to rate
such answers as good. Table 3 and 4 show query, bot response, and GPT-4
evaluation examples.

query |  is there annual fee for card
---|---
bot response |  Yes, there is an annual fee of INR 500 for the credit card. However, this fee will be waived off if your annual spends exceed INR 3.5 Lakhs.
gpt4 eval |  Good
gpt4 reason |  The answer is directly addressing the query and providing the relevant information from the FAQ context. The answer matches with the information in the FAQ context, stating that there is an annual fee of INR 500 for the the credit card, which will be waived off if the annual spends exceed INR 3.5 Lakhs.
Table 3: GPT-4 evaluation for an in-domain query query |  hows weather today
---|---
bot response |  I’m sorry, but I don’t have information about the weather. How can I assist you with the credit card?
gpt4 eval |  Good
gpt4 reason |  The answer admits that it does not have information about the weather and asks how it can assist with the the credit card. The answer is relevant to the query and meets the answerability criteria.
Table 4: GPT-4 evaluation for an OOD query

We evaluated the bot output on a chat session’s sequence of 25 queries.
Queries consisted of English and Hinglish domain queries, OOD queries (e.g.,
hows the weather), greeting/acknowledgment queries, etc. The GPT-4 evaluation
provided 100% accuracy for these answers. A manual verification of evaluations
was done to re-confirm the results. The evaluation indicated that the FAQ bot
is able to respond accurately to user queries.

## Optimizing RAG for FAQ chatbots

For publicly available API-based paid LLMs (e.g., ChatGPT), the cost is
calculated based on the number of input and output tokens. The larger the
number of tokens, the higher the cost. Also, it was observed that the accuracy
of LLMs degraded with more tokens since LLMs are tasked with the additional
task of identifying relevant tokens from the entire context (Liu et al.
2023a). LLM cost can be significant if many users are interacting with the
bot. In this section, we propose two approaches to optimize the cost of LLMs
without trading off the accuracy.

### Reinforcement learning for optimizing RAG

In RAG, we fetch a top-k FAQ context for each query, pass it to an LLM, and
generate the answer. However, there are patterns/sequences of queries for
which not fetching a FAQ context may still lead to a good bot answer. This can
happen under the following scenarios.

  * •

Follow up queries:

For a follow-up query, if the relevant context was fetched with the previous
query, it would already be present with LLM. Hence, not retrieving a FAQ
context for a follow-up query would still lead to the same answer.

  * •

Queries referring to the same FAQ:

FAQs are designed to answer queries regarding a particular topic, e.g., in our
FAQs, all the merchant cashback-related queries and benefits-related queries
get answered from a FAQ ’what are the card benefits’. For the sequence of such
queries referring to the same FAQ, we can fetch FAQ context only for the first
query, and all the subsequent queries can utilize this context. Hence,
fetching FAQ context on every query is unnecessary under such a scenario.

  * •

OOD queries:

LLM is prompted to answer OOD queries with a "do not know" answer. Hence, even
if we fetch FAQ context for OOD queries, LLM is instructed to ignore it.
Hence, fetching a context would not be needed for OOD queries as well.

As can be seen from the list above, there are many cases for which fetching
FAQ context would not be needed to get the same answer as with the usual RAG
pipeline and, therefore, there is a scope to optimize the tokens passed to the
LLM to reduce the cost without trading off the accuracy.

We propose a Reinforcement Learning (RL) based approach to optimize the number
of tokens passed to LLM. Figure 1 shows the proposed approach for policy-based
agent for optimizing RAG. Specifically, we train a policy network to maximize
rewards from a GPT-4 evaluator. The policy network resides outside the RAG
pipeline and tries to optimize it by tuning the policy. We assume that we do
not have access to gradients of the retrieval model or LLM and treat these
components as fixed.

Input to the policy network is the State, which comprises previous queries,
the corresponding actions (encoded as tokens), and the current query. Since
our RAG pipeline uses the last two conversations as history, for the policy
model as well, we maintain the previous two queries as context. The policy
network can take two actions for the current query: [FETCH] and [NO_FETCH]. As
the name suggests, when a policy network chooses a [FETCH] action, a usual RAG
pipeline would be executed. When a policy network chooses a [NO_FETCH] action,
FAQ context is not retrieved, and the current query is directly inputted to
the LLM. The LLM then generates the answer. The current query, answer, and
conversation history is inputted to a GPT-4, which rates the bot answer as
good or bad.

Intuitively, if a policy model chooses a wrong action, in that case, GPT-4
should provide a rating as Bad, e.g., if a policy model chooses a [NO_FETCH]
action for the in-domain query without any earlier context. We observe that it
is indeed the case, and Table 5 shows an example of such a case.

query |  how much cashback on swiggy
---|---
policy action |  [NO_FETCH]
bot response |  You will get 1.5% unlimited cashback on Swiggy with the credit card.
gpt4 eval |  Bad
gpt4 reason |  The given answer is incorrect. According to the FAQ context, the credit card offers 4% unlimited cashback on preferred partners, including Swiggy. The answer provided 1.5% cashback, which is not accurate.
Table 5: GPT-4 evaluation for an in-correct policy action Action | GPT-4 eval | Reward
---|---|---
FETCH | - | 0.1
NO_FETCH | Good | 2
NO_FETCH | Bad | -1
Table 6: Reward shaping: converting GPT-4 evaluations to numeric rewards.

The GPT-4 evaluation rating is converted to the numeric reward ($r$) value as
depicted in Table 6. An intuition behind this reward function is that when a
policy model executes a [NO_FETCH] action and GPT-4 rates the bot response as
Good, it indicates that for the input query not fetching a context still leads
to a good output, possibly because one of the reasons mentioned in the
itemized list. Hence, to promote such actions, we give a high positive reward.
If the policy model plays a [FETCH] action, we give a small positive reward.
If GPT-4 rates the bot response as Bad for either of the actions, we provide a
negative reward to demote such actions, e.g., the policy model gets a negative
reward when it does not fetch context when required. To minimize the cost for
GPT-4 evaluations, we only evaluate the cases where a policy model plays a
[NO_FETCH] action. This is because a RAG pipeline is expected to work well
with [FETCH] action, and we want to understand when executing a [NO_FETCH]
action is favorable. If an [FETCH] action leads to a wrong answer, then
something within a RAG pipeline (embedding model/prompt) needs to be updated,
and a policy model that resides outside the RAG will be unable to correct it.
If the policy model plays a [FETCH] action, we directly assign a small
positive reward without GPT-4 evaluation. Intuitively, we want a policy model
to choose a [FETCH] action unless it is highly confident that a [NO_FETCH]
action will provide a correct answer.

Table 7 shows a sample sequence of state, action, and rewards for the chat
session with two queries [’is there an annual fee?’, ’can you reduce it?’].

$s_{0}$ |  [CLS] is there annual fee [SEP]
---|---
$a_{0}$ |  [FETCH]
$r_{0}$ |  0.1
$s_{1}$ |  [CLS] is there annual fee [SEP] [FETCH] can you reduce it [SEP]
$a_{1}$ |  [NO_FETCH]
$r_{1}$ |  2.
Table 7: Sequence of (State, Action, Reward) for a sample chat session with
two queries

We calculate the cumulative sum of rewards $G_{t}$ for the $t^{th}$ query in a
training chat session using the Eq. 2.

$G_{t}=\sum_{k=0}^{N}\gamma^{k}r_{t+k+1}$ (2)

where $\gamma$ is the _discount factor_ and $N$ indicates the number of
queries in a chat session. We set the value of $\gamma$ to a small value (0.1)
since we care for an immediate reward, i.e., obtaining good output for each
query.

We train the policy model on six chat sessions consisting of 168 queries.
Additional chat sessions are created by randomly shuffling the queries within
a chat session. For each query within a session, we sample a [FETCH] or
[NO_FETCH] action based on the initial policy. The sampled action is executed,
and a bot response is generated. GPT-4 then evaluates the bot response and
provides a rating as Good or Bad, which is then converted to a numeric reward.
We repeat this process multiple times per chat session and create a dataset of
1733 (state, action, rating) tuples.

As a policy model, we use an in-house pre-trained BERT model. The in-house
BERT model has the same architecture as bert-base-uncased, with 12 layers and
an encoding dimension 768. The model is trained using Masked language modeling
(MLM) and Next sentence prediction (NSP) on in-domain text such as product
descriptions, user reviews, etc. We add a linear layer with Softmax activation
on the last layer’s embedding of the [CLS] token to map the State
representation onto a 2-dimensional action space. We add two new tokens to the
vocab representing policy actions, [FETCH] and [NO_FETCH], and randomly
initialize their embedding. The policy network is trained with the policy
gradient loss and entropy regularization loss shown in Eq. 3. Entropy
regularization is shown to help exploration and even leads to better
optimization (Ahmed et al. 2019).

$l_{t}=-\log\pi_{\theta}(a_{t}|s_{t})G_{t}-\lambda\hskip
2.84544ptH(\pi_{\theta}(a_{t}|s_{t}))$ (3)

where $H$ indicates entropy and $\lambda$ indicates the entropy loss weight.
We set $\lambda$ to 0.1 for all our experiments. $s_{t}$ indicates the state
at $t^{th}$ time step and $a_{t}$ indicates action taken at the $t^{th}$ time
step.

We test the policy model on an extensive test chat session of 91 queries,
which includes an interleaved sequence of in-domain queries (how to apply for
card), greeting/acknowledgment queries (hi, ok, cool, thank u), and OOD
queries. To judge the number of token savings obtained with the proposed
optimizations, we compare the number of tokens passed to an LLM with a usual
RAG pipeline to the settings where we introduce the optimizations for RAG. The
number of tokens for LLM calls was calculated using the tiktoken library. With
a usual RAG pipeline, we fetch a FAQ context for each query.

With a policy-based approach, the policy model provides a probability
prediction over actions for each query. We use the Monte Carlo dropout method
to estimate uncertainty and average the probabilities over ten predictions. A
[FETCH] or [NO_FETCH] action is selected based on the averaged probability.

### Similarity threshold on top-1 score (SimThr)

As described earlier, the in-house embedding model trained with infoNCE loss
provides accurate similarity scores for FAQs and discriminate scores w.r.t.
OOD queries. An LLM call can be avoided if the top-1 similarity score is
large. Hence, if the top-1 cosine score is above a pre-defined threshold
(0.92) for a user query, we directly output a corresponding static FAQ answer.
We execute a usual RAG pipeline if a similarity score is less than a
threshold. Note that using such a threshold would not be reliable with the
general-purpose public embedding model.

### Results

Table 8 shows the token saving under different experimental settings. The
threshold-based approach (SimThr) provides ~12% saving on the number of
tokens. Introducing an additional policy-based dynamic context selection
provides ~31% saving in the number of tokens. For all the settings, we
performed a manual quality evaluation of the bot responses. We observe a
slightly lower accuracy for All fetch and All fetch + SimThr setting than the
SimThr + policy setting. In particular, we notice that for All fetch and All
fetch + SimThr setting, for one of the acknowledgment queries (’cool’), the
bot hallucinates and incorrectly outputs an answer for an FAQ. For the same
query and context, a policy-based setting correctly outputs the relevant
message.

Note that even with a limited training set available for policy training, we
achieved significant savings on the number of tokens using the combination of
SimThr and a policy-based approach. Out of the entire token saving, ~61% can
be attributed to the policy model, and ~39% can be attributed to the SimThr
approach.

Setting | # tokens | saving | Acc.
---|---|---|---
All fetch | 167885 | - | 0.989
SimThr | 147147 | ~12% | 0.989
SimThr + policy | 115259 | ~31% | 1.
Table 8: Comparison of token savings with different experimental settings

We also experimented with a different reward shaping, in which, for [NO_FETCH]
action, we give a positive reward on the same scale as the negative reward. We
observed a token saving of ~30% for the same test set, less than the earlier
policy. This indicates that the reward shaping affects the token saving.

Table 9 shows the result of the policy model on the sequence of three queries.
In our FAQ dataset, all merchant cashback-related queries get answered from
the same FAQ ’what are the card benefits’. For the sequence of these cashback-
related queries, the policy model correctly retrieves context only for the
first query, does not retrieve context for subsequent queries, and still
provides the accurate answer. This underlines the effectiveness of the
proposed RL-based approach for FAQ chatbots with RAG.

query |  what cashback on i will get on swiggy
---|---
policy action |  [FETCH]
bot answer |  You will get 4% unlimited cashback on Swiggy with the credit card.
human eval |  Good
query |  what about pvr
policy action |  [NO_FETCH]
bot answer |  You will get 4% unlimited cashback on PVR with the credit card.
human eval |  Good
query |  how much on curefit
policy action |  [NO_FETCH]
bot answer |  You will get 4% unlimited cashback on Curefit with the credit card.
human eval |  Good
Table 9: Policy model predictions on a sequence of queries referring to the
same FAQ

### Comparison with gpt-2 as policy model

We compare the results obtained with the in-house BERT policy model with the
publicly available gpt-2 model as a policy model. For gpt-2 model, apart from
[FETCH] and [NO_FETCH] tokens, we also add [CLS] and [SEP] tokens to the vocab
and randomly initialize their embeddings. For gpt-2, we use the embedding from
the last layer’s [SEP] token and add a linear layer on top of it. All the
remaining experimental settings are the same as the in-house BERT experiment.
Table 10 shows the comparison result. Using the gpt-2 model as the policy
provides ~25% of token saving with the same accuracy. This indicates that the
in-house BERT model performs better with the limited training set due to
domain pre-training.

Setting | # tokens | saving | Acc.
---|---|---|---
SimThr + policy (gpt-2) | 124924 | ~25% | 1.
SimThr + policy (BERT) | 115259 | ~31% | 1.
Table 10: Comparison of token saving with a open gpt-2 as policy model

## Conclusion

In this paper, we described a RAG-based chatbot for answering credit card-
related queries using the domain FAQ dataset. We generated the question
paraphrase data using the public LLM model and trained an in-house embedding
model for retrieval using the infoNCE loss. An in-house model was shown to
perform significantly better than a general-purpose public model in terms of
ranking accuracy and OOD query detection. Further, we noticed that for
specific patterns/sequences of queries, it is not required to fetch the FAQ
context to get a good answer. Assuming a fixed retrieval model and LLM, we
optimize the number of tokens passed to an LLM using Reinforcement Learning.
We trained a policy model that resides external to the RAG and decides whether
to retrieve a context. We used a GPT-4 as the reward model. The RL-based
optimization combined with the similarity threshold led to significant token
savings while slightly improving the accuracy. The proposed policy-based
approach is generic and can be used with any existing RAG pipeline.

## References

  * kha (2023) 2023\.  Augmented Embeddings for Custom Retrievals.  arXiv:2310.05380.
  * Ahmed et al. (2019) Ahmed, Z.; Roux, N. L.; Norouzi, M.; and Schuurmans, D. 2019.  Understanding the impact of entropy on policy optimization.  arXiv:1811.11214.
  * Asai et al. (2023) Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and Hajishirzi, H. 2023.  Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection.  arXiv:2310.11511.
  * Bacciu et al. (2023) Bacciu, A.; Cocunasu, F.; Siciliano, F.; Silvestri, F.; Tonellotto, N.; and Trappolini, G. 2023.  RRAML: Reinforced Retrieval Augmented Machine Learning.
  * Goyal et al. (2021) Goyal, P.; Caron, M.; Lefaudeux, B.; Xu, M.; Wang, P.; Pai, V.; Singh, M.; Liptchinsky, V.; Misra, I.; Joulin, A.; and Bojanowski, P. 2021.  Self-supervised Pretraining of Visual Features in the Wild.  arXiv:2103.01988.
  * Hackl et al. (2023) Hackl, V.; Müller, A. E.; Granitzer, M.; and Sailer, M. 2023.  Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings.  arXiv:2308.02575.
  * Lewis et al. (2021) Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; Küttler, H.; Lewis, M.; tau Yih, W.; Rocktäschel, T.; Riedel, S.; and Kiela, D. 2021.  Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.  arXiv:2005.11401.
  * Li et al. (2022) Li, H.; Su, Y.; Cai, D.; Wang, Y.; and Liu, L. 2022.  A survey on retrieval-augmented text generation.  _arXiv preprint arXiv:2202.01110_.
  * Liu et al. (2023a) Liu, N. F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, M.; Petroni, F.; and Liang, P. 2023a.  Lost in the Middle: How Language Models Use Long Contexts.  arXiv:2307.03172.
  * Liu et al. (2023b) Liu, Y.; Iter, D.; Xu, Y.; Wang, S.; Xu, R.; and Zhu, C. 2023b.  G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment.  arXiv:2303.16634.
  * Shao et al. (2023) Shao, Z.; Gong, Y.; Shen, Y.; Huang, M.; Duan, N.; and Chen, W. 2023.  Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy.  arXiv:2305.15294.
  * van den Oord, Li, and Vinyals (2019) van den Oord, A.; Li, Y.; and Vinyals, O. 2019.  Representation Learning with Contrastive Predictive Coding.  arXiv:1807.03748.
  * Wang et al. (2022) Wang, L.; Yang, N.; Huang, X.; Jiao, B.; Yang, L.; Jiang, D.; Majumder, R.; and Wei, F. 2022.  Text Embeddings by Weakly-Supervised Contrastive Pre-training.  arXiv:2212.03533.


continuous paths imply that these two processes are indistinguishable. Thus,
the inequality (4.44) is nothing else than the inequality (2.26).

It remains us to show that (4.42) holds. In fact, it results from Lemmata 4.2,
4.5, 4.6, 4.9, 4.15 and the inequality:

$\sum_{\mathbf{j}\in\mathbb{Z}^{d}\,:\,\displaystyle\max_{\ell\in[\\![1,d]\\!]}j_{\ell}\geq
J}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\|X_{\mathbf{j}}\|_{I,\infty}\leq\sum_{n=1}^{d}\left(\mathcal{M}_{n,J}+\sum_{m=0}^{3}\mathcal{H}_{n,J}^{m}\right),$

which is obtained by using (4.41), the triangular inequality, standard
computations, and the definitions of the random variables $\mathcal{M}_{n,J}$,
and $\mathcal{H}_{n,J}^{m}$ with $m\in[\\![0,3]\\!]$. ∎

##  5 Proof of Theorem 2.14

In this final Section, we aim at proving Theorem 2.14. We will need a number
of intermediary results which mainly consist in bounding in convenient ways
well-chosen parts of the random series in (2.27). In fact, most of the job has
already been done in the previous Section 4 but there still remain some parts
of the series which need to be conveniently bounded. It is the purpose of the
following lemmata. We mention that the event $\Omega^{*}$ of probability $1$
(see Lemma 2.15) will appear in their statements.

######  Lemma 5.1.

Let $T>2$ and $L>3/2$ be two fixed real numbers. There exits a positive almost
surely finite random variables $C$ such that, for all $n\in[\\![1,d]\\!]$ and
$N\in\mathbb{N}$, on $\Omega^{*}$, the random variable

$\displaystyle\mathcal{L}^{1}_{n,N}:=\sum_{\mathbf{j}\in\mathbb{Z}^{d}\,:\,\max_{\ell\in[\\![1,d]\\!]}j_{\ell}<N}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\times\ldots$
$\displaystyle\qquad\ldots\times\sum_{|k_{n}|>2^{N+1}T}\sum_{\begin{subarray}{c}k_{\ell}\in\mathbb{Z}\\\
\ell\in[\\![1,d]\\!]\setminus\\{n\\}\end{subarray}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
(5.1)

is bounded from above by
$C2^{-N(h_{1}+\cdots+h_{d}+L-d-1)}N^{\frac{d}{2}}\log(3+N)^{\frac{d}{2}}.$

###### Proof.

Let us fix $N\in\mathbb{N}$ and $\mathbf{j}\in\mathbb{Z}^{d}$ such that
$\max_{\ell\in[\\![1,d]\\!]}j_{\ell}<N$. Using the definition (4.1), the fast
decay property (1.3), the bound (2.32), the triangular inequality,
inequalities (B.1) and (B.3) and the fact that the function
$y\mapsto(2+y)^{-L}\sqrt{\log(2+y)}$ is decreasing on $\mathbb{R}_{+}$, we get

$\displaystyle\sum_{|k_{n}|>2^{N+1}T}\sum_{\begin{subarray}{c}k_{\ell}\in\mathbb{Z}\\\
\ell\in[\\![1,d]\\!]\setminus\\{n\\}\end{subarray}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
$\displaystyle\leq
C_{0}\int_{0}^{T}\sum_{|k_{n}|>2^{N+1}T}\frac{\sqrt{\log(3+|j_{n}|+|k_{n}|)}}{(3+|2^{j_{n}}s-k_{n}|)^{L}}\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\begin{subarray}{c}\ell=1\\\ \ell\neq
n\end{subarray}}^{d}\left(\sum_{k_{\ell}\in\mathbb{Z}}\frac{\sqrt{\log(3+|j_{\ell}|+|k_{\ell}|)}}{(3+|2^{j_{\ell}}s-k_{\ell}|)^{L}}\right)\,ds$
$\displaystyle\leq
C_{1}\int_{0}^{T}\sum_{|k_{n}|>2^{N+1}T}\frac{\sqrt{\log(3+|j_{n}|+|k_{n}|)}}{(3+|k_{n}|-2^{j_{n}}s)^{L}}\prod_{\begin{subarray}{c}\ell=1\\\
\ell\neq
n\end{subarray}}^{d}\left(\sqrt{\log(3+|j_{\ell}|+2^{j_{\ell}}s)}\right)\,ds$
$\displaystyle\leq
C_{1}T2^{L}\sqrt{\log(3+|j_{n}|)}\left(\prod_{\begin{subarray}{c}\ell=1\\\
\ell\neq
n\end{subarray}}^{d}\sqrt{\log(3+|j_{\ell}|+2^{j_{\ell}}T)}\right)\times\ldots$
$\displaystyle\qquad\ldots\times\left(\sum_{|k_{n}|>2^{N+1}T}\frac{\sqrt{\log(3+|k_{n}|)}}{(3+|k_{n}|)^{L}}\right)$
$\displaystyle\leq
C_{1}T2^{L+1}\sqrt{\log(3+|j_{n}|)}\left(\prod_{\begin{subarray}{c}\ell=1\\\
\ell\neq
n\end{subarray}}^{d}\sqrt{\log(3+|j_{\ell}|+2^{j_{\ell}}T)}\right)\times\ldots$
$\displaystyle\qquad\ldots\times\left(\int_{2^{N+1}T}^{+\infty}\frac{\sqrt{\log(2+y)}}{(2+y)^{L}}\right)$
$\displaystyle\leq
C_{2}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}N^{\frac{d}{2}}2^{-N(L-1)},$
(5.2)

where $C_{0}$, $C_{1}$ and $C_{2}$ are positive almost surely finite random
variables not depending on $\mathbf{j}$ and $N$. It follows from (5.2) that

$\displaystyle\mathcal{L}^{1}_{n,N}$ $\displaystyle\leq
C_{2}N^{\frac{d}{2}}2^{-N(L-1)}\sum_{j_{1}=-\infty}^{N-1}\cdots\sum_{j_{d}=-\infty}^{N-1}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}$
$\displaystyle\leq
C_{3}N^{\frac{d}{2}}\log(3+N)^{\frac{d}{2}}2^{-N(h_{1}+\cdots+h_{d}+L-d-1)},$

where $C_{3}$ is a positive almost surely finite random variable not depending
on $N$. ∎

######  Lemma 5.2.

Let $T>2$, $L>1$ and $g>0$ be three fixed real numbers. There exits a positive
almost surely finite random variable $C$ such that, for all
$n\in[\\![1,d]\\!]$ and $N\in\mathbb{N}$, on $\Omega^{*}$, the random variable

$\displaystyle\mathcal{L}^{2}_{n,N}:=\sum_{\mathbf{j}\in-\mathbb{N}^{d}}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\sum_{|k_{n}|>2^{Ng}}\sum_{\begin{subarray}{c}k_{\ell}\in\mathbb{Z}\\\
\ell\in[\\![1,d]\\!],\ell\neq
n\end{subarray}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
(5.3)

is bounded from above by $C2^{-N(L-1)g}\sqrt{N}.$

###### Proof.

Let ux fix $N\in\mathbb{N}$ and $\mathbf{j}\in-\mathbb{N}^{d}$. Using the
definition (4.1), the fast decay property (1.3), the bound (2.32), the
inequality (B.1), the triangular inequality and the fact that the function
$y\mapsto(2+y)^{-L}\sqrt{\log(2+y)}$ is decreasing on $\mathbb{R}_{+}$, we get

$\displaystyle\sum_{|k_{n}|>2^{Ng}}\sum_{\begin{subarray}{c}k_{\ell}\in\mathbb{Z}\\\
\ell\in[\\![1,d]\\!]\setminus\\{n\\}\end{subarray}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
$\displaystyle\leq
C_{0}\int_{0}^{T}\sum_{|k_{n}|>2^{Ng}}\frac{\sqrt{\log(3+|j_{n}|+|k_{n}|)}}{(3+T+|2^{j_{n}}s-k_{n}|)^{L}}\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\begin{subarray}{c}\ell=1\\\ \ell\neq
n\end{subarray}}^{d}\left(\sum_{k_{\ell}\in\mathbb{Z}}\frac{\sqrt{\log(3+|j_{\ell}|+|k_{\ell}|)}}{(3+T+|2^{j_{\ell}}s-k_{\ell}|)^{L}}\right)\,ds$
$\displaystyle\leq
C_{0}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}\int_{0}^{T}\sum_{|k_{n}|>2^{Ng}}\frac{\sqrt{\log(3+|k_{n}|)}}{(3+|k_{n}|)^{L}}\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\begin{subarray}{c}\ell=1\\\ \ell\neq
n\end{subarray}}^{d}\left(\sum_{k_{\ell}\in\mathbb{Z}}\frac{\sqrt{\log(3+|k_{\ell}|)}}{(3+|k_{\ell}|)^{L}}\right)\,ds$
$\displaystyle\leq
C_{1}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}\sum_{|k_{n}|>2^{Ng}}\frac{\sqrt{\log(3+|k_{n}|)}}{(3+|k_{n}|)^{L}}$
$\displaystyle\leq
2C_{1}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}\int_{2^{Ng}}^{+\infty}\frac{\sqrt{\log(2+y)}}{(2+y)^{L}}$
$\displaystyle\leq
C_{2}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}2^{-N(L-1)g}\sqrt{N},$ (5.4)

where $C_{0}$, $C_{1}$ and $C_{2}$ are positive almost surely finite random
variables not depending on $\mathbf{j}$ and $N$. It follows from (5.4) that

$\displaystyle\mathcal{L}^{2}_{n,N}$ $\displaystyle\leq
C_{2}2^{-N(L-1)g}\sqrt{N}\sum_{\mathbf{j}\in-\mathbb{N}^{d}}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\sqrt{\prod_{\ell=1}^{d}\log(3+|j_{\ell}|)}$
$\displaystyle\leq C_{3}2^{-N(L-1)g}\sqrt{N},$

where $C_{3}$ is a positive almost surely finite random variables not
depending on $N$. ∎

######  Definition 5.3.

If $b>0$ is a fixed real numbers, we define the set of boolean vector

$\mathcal{B}:=\Big{\\{}v=(v_{l})_{\ell\in[\\![1,d]\\!]}\in\\{0,1\\}^{d}:\exists\,(l^{\prime},l^{\prime\prime})\in[\\![1,d]\\!]^{2}\text{
such that }l^{\prime}\neq l^{\prime\prime}\text{ and }v_{l^{\prime}}\neq
v_{l^{\prime\prime}}\Big{\\}}$

and, for all $v\in\mathcal{B}$ and $N\in\mathbb{N}$, the subset of
$\mathbb{Z}^{d}$

$\gimel_{v,N}^{3}:=\Big{\\{}\mathbf{j}\in\mathbb{Z}^{d}\,:\,\forall\,\ell\in[\\![1,d]\\!],j_{\ell}\in[\\![0,N]\\!]\text{
if }v_{\ell}=0\text{ and, otherwise, }j_{\ell}\leq-2^{Nb}\Big{\\}}.$

######  Lemma 5.4.

Let $T>2$ and $b>0$ be two fixed real numbers. There exits a positive almost
surely finite random variable $C$ such that, for all $v\in\mathcal{B}$ and
$N\in\mathbb{N}$, on $\Omega^{*}$, the random variable

$\displaystyle\mathcal{L}^{3}_{v,N}:=\sum_{\mathbf{j}\in\gimel_{v,N}^{3}}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\sum_{\mathbf{k}\in\mathbb{Z}^{d}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
(5.5)

is bounded from above by
$CN^{\frac{d}{2}}2^{N\left(\sum_{(\ell\,:\,v_{\ell}=0)}(1-h_{\ell})\right)-2^{Nb}\left(\sum_{(\ell^{\prime}\,:\,v_{\ell^{\prime}}=1)}(1-h_{\ell^{\prime}})\right)}.$

###### Proof.

Let ux fix $N\in\mathbb{N}$ and $\mathbf{j}\in\gimel_{v,N}^{3}$. Using the
definition (4.1), the fast decay property (1.3), the bound (2.32), the
inequality (B.1), the triangular inequality and the inequalities (B.3) and
(B.2), we get

$\displaystyle\sum_{\mathbf{k}\in\mathbb{Z}^{d}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
$\displaystyle\leq
C_{0}\int_{0}^{T}\prod_{\ell\,:\,v_{\ell}=0}\left(\frac{\sqrt{\log(3+j_{\ell}+|k_{\ell}|)}}{(3+|2^{j_{\ell}}s-k_{\ell}|)^{2}}\right)\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\ell^{\prime}\,:\,v_{\ell^{\prime}}=1}\left(\frac{\sqrt{\log(3+|j_{\ell^{\prime}}|+|k_{\ell^{\prime}}|)}}{(3+|2^{j_{\ell^{\prime}}}s-k_{\ell^{\prime}}|)^{2}}\right)\,ds$
$\displaystyle\leq
C_{0}\int_{0}^{T}\prod_{\ell\,:\,v_{\ell}=0}\left(\sqrt{\log(3+j_{\ell}+2^{j_{\ell}}T)}\right)\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\ell^{\prime}\,:\,v_{\ell^{\prime}}=1}\left(\frac{\sqrt{\log(3+|j_{\ell^{\prime}}|)}\sqrt{\log(3+|k_{\ell^{\prime}}|)}}{(3+|k_{\ell^{\prime}}|)^{2}}\right)\,ds$
$\displaystyle\leq
C_{1}N^{\\#\\{\ell\,:\,v_{\ell}=0\\}/2}\prod_{\ell^{\prime}\,:\,v_{\ell^{\prime}}=1}\left(\sqrt{\log(3+|j_{\ell^{\prime}}|)}\right),$
(5.6)

where $C_{0}$ and $C_{1}$ are positive almost surely finite random variables
not depending on $\mathbf{j}$ and $N$. It follows from (5.6) that

$\displaystyle\mathcal{L}^{3}_{v,N}$ $\displaystyle\leq
C_{1}N^{\\#\\{\ell\,:\,v_{\ell}=0\\}/2}\prod_{\ell\,:\,v_{\ell}=0}\left(\sum_{j_{\ell}=0}^{N}2^{j_{\ell}(1-h_{\ell})}\right)\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\ell^{\prime}\,:\,v_{\ell^{\prime}}=1}\left(\sum_{j_{\ell^{\prime}}=-\infty}^{\lfloor-2^{Nb}\rfloor}2^{j_{\ell^{\prime}}(1-h_{\ell^{\prime}})}\sqrt{\log(3+|j_{\ell^{\prime}}|)}\right)$
$\displaystyle\leq
C_{2}N^{\frac{d}{2}}2^{N\left(\sum_{(\ell\,:\,v_{\ell}=0)}(1-h_{\ell})\right)-2^{Nb}\left(\sum_{(\ell^{\prime}\,:\,v_{\ell^{\prime}}=1)}(1-h_{\ell^{\prime}})\right)},$

where $C_{2}$ is a positive almost surely finite random variable not depending
on $N$. ∎

######  Definition 5.5.

Let $b^{\prime}>0$ be a fixed real number. For all boolean vector
$v=(v_{l})_{\ell\in[\\![1,d]\\!]}\in\\{0,1\\}^{d}$ and $N\in\mathbb{N}$, we
define the subset of $\mathbb{Z}^{d}$

$\gimel_{v,N}^{4}:=\big{\\{}j\in-\mathbb{N}^{d}\,:\,\forall\,\ell\in[\\![1,d]\\!],\,j_{\ell}\leq-2^{Nb^{\prime}}\text{
if }v_{\ell}=1\big{\\}}.$

######  Lemma 5.6.

Let $T>2$ and $b^{\prime}>0$ be two fixed real numbers. There exits a positive
almost surely finite random variable $C$ such that, for all
$v\in\\{0,1\\}^{d}$ and $N\in\mathbb{N}$, on $\Omega^{*}$, the random variable

$\displaystyle\mathcal{L}^{4}_{v,N}:=\sum_{\mathbf{j}\in\gimel_{v,N}^{4}}2^{j_{1}(1-h_{1})+\cdots+j_{d}(1-h_{d})}\sum_{\mathbf{k}\in\mathbb{Z}^{d}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}$
(5.7)

is bounded from above by
$CN^{\frac{d}{2}}2^{-2^{Nb^{\prime}}\left(\sum_{(\ell\,:\,v_{\ell}=1)}(1-h_{\ell})\right)}$.

###### Proof.

Let us fix $N\in\mathbb{N}$ and $\mathbf{j}\in\gimel_{v,N}^{4}$. Using the
definition (4.1), the fast decay property (1.3), the bound (2.32), the
inequality (B.1) and the triangular inequality, we get

$\displaystyle\sum_{\mathbf{k}\in\mathbb{Z}^{d}}\left|\varepsilon_{\mathbf{j},\mathbf{k}}\right|\sup_{t\in[0,T]}\left\\{\left|\mathcal{A}_{\mathbf{j},\mathbf{k}}(t)\right|\right\\}\leq
C_{0}\int_{0}^{T}\prod_{\ell=1}^{d}\left(\sum_{k_{\ell}\in\mathbb{Z}}\frac{\sqrt{\log(3+|j_{\ell}|+|k_{\ell}|)}}{(3+|2^{j_{\ell}}s-k_{\ell}|)^{2}}\right)\,ds$
$\displaystyle\leq
C_{0}\int_{0}^{T}\prod_{\ell=1}^{d}\left(\sum_{k_{\ell}\in\mathbb{Z}}\frac{\sqrt{\log(3+|j_{\ell}|)}\sqrt{\log(3+|k_{\ell}|)}}{(3+|k_{\ell}|)^{2}}\right)\,ds$
$\displaystyle\leq C_{1}\prod_{\ell=1}^{d}\sqrt{\log(3+|j_{\ell}|)},$ (5.8)

where $C_{0}$ and $C_{1}$ are positive almost surely finite random variables
not depending on $\mathbf{j}$ and $N$. It follows from (5.8) that

$\displaystyle\mathcal{L}^{4}_{v,N}$ $\displaystyle\leq
C_{1}\prod_{\ell\,:\,v_{\ell}=1}\left(\sum_{j_{\ell}=-\infty}^{\lfloor-2^{Nb^{\prime}}\rfloor}2^{j_{\ell}(1-h_{\ell})}\sqrt{\log(3+|j_{\ell}|)}\right)\times\ldots$
$\displaystyle\qquad\ldots\times\prod_{\ell^{\prime}\,:\,v_{\ell^{\prime}}=0}\left(\sum_{j_{\ell^{\prime}}=-\infty}^{-1}2^{j_{\ell^{\prime}}(1-h_{\ell^{\prime}})}\sqrt{\log(3+|j_{\ell^{\prime}}|)}\right)$
$\displaystyle\leq
C_{2}N^{\frac{d}{2}}2^{-2^{Nb^{\prime}}\left(\sum_{(\ell\,:\,v_{\ell}=1)}(1-h_{\ell})\right)},$

where $C_{2}$ is a positive almost surely finite random variable not depending
on $N$. ∎

We are now in position to complete the proof of Theorem 2.14.

###### End of the Proof of Theorem 2.14.

The interval $[0,T]$ in statement of Theorem 2.14 is denoted by $I$ in this
proof. Let $\widetilde{\Omega}$ be the same event of probability $1$ as in the
proof of Theorem 2.12; recall that it is defined as:
$\widetilde{\Omega}=\Omega^{*}\cap\Omega^{**}$, where $\Omega^{*}$ and
$\Omega^{**}$ are as in Lemmata 2.15 and 4.14. Next, observe that for proving
Theorem 2.14 it is enough to show that there exists a positive almost surely
finite random variable $C$ such that, on $\widetilde{\Omega}$, we have, for
all $N,P\in\mathbb{N}$,

$\big{\|}\widetilde{X}^{(d)}_{\mathbf{h},N+P}-\widetilde{X}^{(d)}_{\mathbf{h},N}\big{\|}_{I,\infty}\leq
CN^{\frac{d}{2}}2^{-N(h_{1}+\cdots+h_{d}-d+1/2)},$ (5.9)

where, for all fixed $\omega\in\Omega$, the continuous function
$\widetilde{X}^{(d)}_{\mathbf{h},N}(\cdot,\omega)$ is defined through (2.28).
Indeed, assuming that (5.9) is true, then it turns out that, for each fixed
$\omega\in\widetilde{\Omega}$, the sequence of functions
$\big{(}\widetilde{X}^{(d)}_{\mathbf{h},N}(\cdot,\omega)\big{)}_{N\in\mathbb{N}}$
is a Cauchy sequence in the Banach space of the real-valued continuous
functions over $I$, equipped with its natural norm $\|\cdot\|_{I,\infty}$.
Therefore, it converges, for this norm, to a continuous function over $I$
denoted by $\widetilde{X}^{(d)}_{\mathbf{h}}(\cdot,\omega)$. On the other
hand, when $\omega\in\Omega\setminus\widetilde{\Omega}$ we set
$\widetilde{X}^{(d)}_{\mathbf{h}}(t,\omega)=0$, for all $t\in I$. Next,
observe that, in view of the previous definition of the stochastic process
$\\{\widetilde{X}^{(d)}_{\mathbf{h}}(t)\\}_{t\in I}$ we have, for all $t\in
I$, almost surely,

$X^{(d)}_{\mathbf{h}}(t)=\widetilde{X}^{(d)}_{\mathbf{h}}(t),$ (5.10)

since we know from (2.28) and (2.27), that, for each fixed
$t\in\mathbb{R}_{+}$ (and in particular for $t\in I$), the sequence of random
variables $(\widetilde{X}^{(d)}_{\mathbf{h},N}(t,\omega))_{N\in\mathbb{N}}$
converges to $X^{(d)}_{\mathbf{h}}(t)$ in $L^{2}(\Omega)$. Next, using the
fact that the two stochastic processes $\\{X^{(d)}_{\mathbf{h}}(t)\\}_{t\in
I}$ and $\\{\widetilde{X}^{(d)}_{\mathbf{h}}(t)\\}_{t\in I}$ have continuous
paths, we can derive from the almost sure equality (5.10) that these two
processes are indistinguishable. Thus, letting $P$ in (5.9) tends to $+\infty$
, we obtain (2.29).

It remains us to show that (5.9) holds. Using the inclusion

$D_{j}^{1}(t)\subseteq\\{k\in\mathbb{Z}\,:\,|k|\leq
2^{N+P+1}T\\},\quad\mbox{for all $t\in I:=[0,T]$ and $N\leq j\leq N+P$,}$

and the triangle inequality, we get that

$\displaystyle\|\widetilde{X}^{(d)}_{\mathbf{h},N+P}-\widetilde{X}^{(d)}_{\mathbf{h},N}\|_{I,\infty}$
$\displaystyle\leq\sum_{n=1}^{d}\left(\mathcal{M}_{n,N}+\sum_{m=0}^{3}\mathcal{H}_{n,N}^{m}+\sum_{m=1}^{2}\mathcal{L}_{n,N}^{m}\right)+\sum_{v\in\mathcal{B}}\mathcal{L}_{v,N}^{3}+\sum_{v\in\\{0,1\\}^{d}}\mathcal{L}_{v,N}^{4}.$

Then the fact that (5.9) is satisfied follows from Lemmata 4.2, 4.5, 4.6,
4.9,4.15, 5.1, 5.2, 5.4 and 5.6. ∎

Acknowledgement. The three authors of this work are members of the GDR 3475
(Analyse Multifractale et Autosimilarité) which partially supports them.
Antoine Ayache is also partially supported by the Labex CEMPI
(ANR-11-LABX-0007-01) and the Australian Research Council’s Discovery Projects
funding scheme (project number DP220101680). Laurent Loosveldt is supported by
the FNR OPEN grant APOGEe at University of Luxembourg. The first results of
this paper were formulated during a visit of Antoine Ayache at the University
of Luxembourg, also funded by the FNR OPEN grant APOGEe.

##  Appendix A Some facts concerning multiple Wiener integrals

In this section, mainly we give the proof of the crucial equality (2.21). This
proof relies on some fundamental facts concerning multiple Wiener integrals.
We refer to the two books [22, 23] for detailed presentations of such
stochastic integrals and many other related topics (Wiener chaoses, Malliavin
calculus, and so on). We recall that a function $f\in L^{2}(\mathbb{R}^{n})$
is said to be symmetric if, for all $\sigma\in\mathfrak{S}_{n}$ (the set of
permutations of $[\\![1,n]\\!]=\\{1,\ldots,n\\}$) and for Lebesgue almost
every $(x_{1},\ldots,x_{n})\in\mathbb{R}^{n}$, one has
$f(x_{\sigma(1)},\ldots,x_{\sigma(n)})=f(x_{1},\ldots,x_{n})$. In other words,
$f\in L^{2}(\mathbb{R}^{n})$ is symmetric if and only if it is almost
everywhere equal to its canonical symmetrization $\widetilde{f}$ defined, for
all $(x_{1},\ldots,x_{n})\in\mathbb{R}^{n}$, as:

$\widetilde{f}(x_{1},\ldots,x_{n}):=\frac{1}{n!}\sum_{\sigma\in\mathfrak{S}_{n}}f(x_{\sigma(1)},\ldots,x_{\sigma(n)}).$
(A.1)

We point out that a very fundamental property of multiple integrals is that

$I_{n}(f)=I_{n}(\widetilde{f}),\quad\mbox{for all $f\in
L^{2}(\mathbb{R}^{n})$.}$ (A.2)

For proving the equality (2.21), we will make use of the so-called product
formula for multiple Wiener integrals [23, Proposition 1.1.3]. In order to
give this important formula, first we need the following definition: let $m$
and $n$ be two arbitrary positive integers, if $f\in L^{2}(\mathbb{R}^{m})$
and $g\in L^{2}(\mathbb{R}^{n})$ are symmetric functions and
$r\in[\\![0,m\wedge n]\\!]$, the contraction $f\otimes_{r}g$ is the
$L^{2}(\mathbb{R}^{m+n-2r})$ function defined, for all
$(x_{1},\ldots,x_{m+n-2r})\in\mathbb{R}^{m+n-2r}$, through the Lebesgue
integral

$\displaystyle(f\otimes_{r}g)(x_{1},\ldots,x_{m+n-2r})$
$\displaystyle:=\int_{\mathbb{R}^{r}}f(x_{1},\ldots,x_{m-r},s_{1},\ldots,s_{r})g(x_{m-r+1},\ldots,x_{m+n-2r},s_{1},\ldots,s_{r})\,ds_{1}\ldots
ds_{r}\,,$

with the convention that $f\otimes_{0}g:=f\otimes g$, which means that
$f\otimes_{0}g$ is the usual tensor product of $f$ and $g$; also notice that
when $m=n$, then $f\otimes_{n}g$ is identified with the Lebesgue integral
$\int_{\mathbb{R}^{n}}fg$. Using, the previous definition, one can write the
product formula for multiple Wiener integrals in the following way: for each
positive integers $m$ and $n$, and for every symmetric functions $f\in
L^{2}(\mathbb{R}^{m})$ and $g\in L^{2}(\mathbb{R}^{n})$, one has

$I_{m}(f)I_{n}(g)=\sum_{r=0}^{m\wedge
n}r!\binom{m}{r}\binom{n}{r}I_{m+n-2r}(f\otimes_{r}g),$ (A.3)

where, for $p=m$ or $p=n$, the quantity $\binom{p}{r}$ is the usual binomial
coefficient

$\binom{p}{r}:=\frac{p!}{r!(p-r)!}\,.$

For proving the equality (2.21), we will also make use of the following
fundamental result, which, for instance corresponds to [22, Theorem 2.7.7].

######  Theorem A.1.

Let $f\in L^{2}(\mathbb{R})$ be such that $\|f\|_{L^{2}(\mathbb{R})}=1$. For
all positive integer $n$, let $H_{n}$ the Hermite polynomial of degree $n$.
Then, one has

$H_{n}\left(I_{1}(f)\right)=I_{n}(f^{\otimes_{n}}).$

We are now in position to prove the equality (2.21)

###### Proof of the equality (2.21).

It follows from Theorem A.1 that

$\prod_{\ell=1}^{p}H_{n_{\ell}}\big{(}I_{1}(\varphi_{\ell})\big{)}=\prod_{\ell=1}^{p}I_{n_{\ell}}\left(\varphi_{\ell}^{\otimes_{n_{\ell}}}\right),$

and thus, it remains to show

$\prod_{\ell=1}^{p}I_{n_{\ell}}\left(\varphi_{\ell}^{\otimes_{n_{\ell}}}\right)=I_{n_{1}+\cdots+n_{p}}\left(\bigotimes_{\ell=1}^{p}\varphi_{\ell}^{\otimes_{n_{\ell}}}\right).$
(A.4)

We proceed by induction on the positive integer $p$. It is clear that (A.4) is
satisfied when $p=1$. So from now on, we assume that $p\geq 2$ and that

$\prod_{\ell=1}^{p-1}I_{n_{\ell}}\left(\varphi_{\ell}^{\otimes_{n_{\ell}}}\right)=I_{n_{1}+\cdots+n_{p-1}}\left(\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}\right).$

Then, setting $n=n_{1}+\cdots+n_{p-1}$ and $d=n_{1}+\cdots+n_{p}=n+n_{p}$, we
can derive from the product formula (A.3) that

$\displaystyle\prod_{\ell=1}^{p}I_{n_{\ell}}\left(\varphi_{\ell}^{\otimes_{n_{\ell}}}\right)$
$\displaystyle=I_{n}\left(\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}\right)I_{n_{p}}\left(\varphi_{p}\right)$
$\displaystyle=\sum_{r=0}^{n\wedge
n_{p}}r!\binom{n}{r}\binom{n_{p}}{r}I_{n+n_{p}-2r}\left(\widetilde{\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\otimes_{r}\varphi_{p}^{\otimes_{n_{p}}}\right)$
$\displaystyle=I_{d}\left(\widetilde{\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\otimes\varphi_{p}^{\otimes_{n_{p}}}\right)=I_{d}\left(\mathop{\vbox{\halign{#\cr\kern
3.0pt\cr$\braceld\leaders\vrule
height=0.39998pt,depth=0.0pt\hfill\braceru$\crcr\kern
3.0pt\nointerlineskip\cr$\hfil\displaystyle{\widetilde{\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\otimes\varphi_{p}^{\otimes_{n_{p}}}}\hfil$\crcr}}}\limits\right).$
(A.7)

Notice that the third equality in (A.7) results from the equality

$\widetilde{\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\otimes_{r}\varphi_{p}^{\otimes_{n_{p}}}=0,\quad\mbox{for
all $r\in[\\![1,n]\\!]$,}$

which is a consequence of the orthonormality of the system
$(\varphi_{\ell})_{\ell=1}^{p}$. Also notice that the last equality in (A.7)
results from (A.2). Next observe that, in view of (A.7) and (A.2), in order to
show that (A.4) holds, it remains us to prove that

$\mathop{\vbox{\halign{#\cr\kern 3.0pt\cr$\braceld\leaders\vrule
height=0.39998pt,depth=0.0pt\hfill\braceru$\crcr\kern
3.0pt\nointerlineskip\cr$\hfil\displaystyle{\widetilde{\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\otimes\varphi_{p}^{\otimes_{n_{p}}}}\hfil$\crcr}}}\limits=\mathop{\vbox{\halign{#\cr\kern
3.0pt\cr$\braceld\leaders\vrule
height=0.39998pt,depth=0.0pt\hfill\braceru$\crcr\kern
3.0pt\nointerlineskip\cr$\hfil\displaystyle{\bigotimes_{\ell=1}^{d}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\hfil$\crcr}}}\limits.$
(A.8)

Notice that any arbitrary permutation $\sigma\in\mathfrak{S}_{n}$ can be
extended in a natural way into a permutation
$\check{\sigma}\in\mathfrak{S}_{d}$ defined, for all $i\in\\{1,\ldots,n\\}$,
as $\check{\sigma}(i)=\sigma(i)$, and for, each $i\in\\{n+1,\ldots,d\\}$, as
$\check{\sigma}(i)=i$. Thus, using (A.1), the latter notation and the fact
that the composition map $\nu\mapsto\nu o\,\check{\sigma}$ is bijection from
$\mathfrak{S}_{d}$ to itself, one gets, for all
$(x_{1},\ldots,x_{d})\in\mathbb{R}^{d}$, that

$\displaystyle\left(\mathop{\vbox{\halign{#\cr\kern
3.0pt\cr$\braceld\leaders\vrule
height=0.39998pt,depth=0.0pt\hfill\braceru$\crcr\kern
3.0pt\nointerlineskip\cr$\hfil\displaystyle{\widetilde{\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\otimes\varphi_{p}^{\otimes_{n_{p}}}}\hfil$\crcr}}}\limits\,\right)(x_{1},\ldots,x_{d})$
$\displaystyle=\frac{1}{d!}\frac{1}{n!}\sum_{\sigma\in\mathfrak{S}_{n}}\sum_{\nu\in\mathfrak{S}_{d}}\bigotimes_{\ell=1}^{p-1}\varphi_{\ell}^{\otimes_{n_{\ell}}}(x_{\nu(\sigma(1))},\ldots,x_{\nu(\sigma(n))})\otimes\varphi_{p}^{\otimes_{n_{p}}}(x_{\nu(n+1)},\ldots,x_{\nu(d)})$
$\displaystyle=\frac{1}{d!}\frac{1}{n!}\sum_{\sigma\in\mathfrak{S}_{n}}\sum_{\nu\in\mathfrak{S}_{d}}\bigotimes_{\ell=1}^{p}\varphi_{\ell}^{\otimes_{n_{\ell}}}(x_{\nu
o\,\check{\sigma}(1)},\ldots,x_{\nu o\,\check{\sigma}(d)})$
$\displaystyle=\frac{1}{d!}\frac{1}{n!}\sum_{\sigma\in\mathfrak{S}_{n}}\sum_{\nu^{\prime}\in\mathfrak{S}_{d}}\bigotimes_{\ell=1}^{p}\varphi_{\ell}^{\otimes_{n_{\ell}}}(x_{\nu^{\prime}(1)},\ldots,x_{\nu^{\prime}(d)})$
$\displaystyle=\frac{1}{d!}\sum_{\nu^{\prime}\in\mathfrak{S}_{d}}\bigotimes_{\ell=1}^{p}\varphi_{\ell}^{\otimes_{n_{\ell}}}(x_{\nu^{\prime}(1)},\ldots,x_{\nu^{\prime}(d)})$
$\displaystyle=\left(\mathop{\vbox{\halign{#\cr\kern
3.0pt\cr$\braceld\leaders\vrule
height=0.39998pt,depth=0.0pt\hfill\braceru$\crcr\kern
3.0pt\nointerlineskip\cr$\hfil\displaystyle{\bigotimes_{\ell=1}^{p}\varphi_{\ell}^{\otimes_{n_{\ell}}}}\hfil$\crcr}}}\limits\,\right)(x_{1},\ldots,x_{d}).$

∎

##  Appendix B Some useful lemmata

The proofs of the following lemmata, which are extensively used in our
articles, can be found in [2].

######  Lemma B.1.

For all $(x,y)\in\mathbb{R}_{+}^{2}$, we have

$\log(3+x+y)\leq\log(3+x)\log(3+y).$ (B.1)

Moreover, for each fixed positive real number $T$, there exists a constant
$c>0$ such that, for every $x\in\mathbb{R}_{+}$, we have

$\log(3+x+2^{x}T)\leq c(1+x).$ (B.2)

######  Lemma B.2.

For each fixed real number $L>1$, there exists a constant $c>0$ such that, for
all $j\in\mathbb{Z}$ and for all $s\in\mathbb{R}$, we have

$\sum_{k\in\mathbb{Z}}\frac{\sqrt{\log(3+|j|+|k|)}}{\big{(}3+|2^{j}s-k|\big{)}^{L}}\leq
c\sqrt{\log\big{(}3+|j|+2^{j}|s|\big{)}}.$ (B.3)

######  Lemma B.3.

For each fixed real number $L>1$, there exists a constant $c>0$ such that, for
all $t\in\mathbb{R}_{+}$, for all $s\in[0,t]$ and for all $j\in\mathbb{N}$, we
have

$\sum_{k\in D_{j}^{3}(t)}\frac{\sqrt{\log(3+|j|+|k|)}}{(3+|2^{j}s-k|)^{L}}\leq
c(1+j)2^{-j(L-1)(1-a)}\sqrt{\log(3+t)},$ (B.4)

where $D_{j}^{3}(t)$ is the infinite subset of $\mathbb{Z}$ defined through
(4.12).

## References

  * [1] Patrice Abry and Fabrice Sellan.  The wavelet-based synthesis for fractional Brownian motion proposed by F. Sellan and Y. Meyer: remarks and fast implementation.  Appl. Comput. Harmon. Anal., 3(4):377–383, 1996.
  * [2] Antoine Ayache and Yassine Esmili.  Wavelet-type expansion of the generalized Rosenblatt process and its rate of convergence.  J. Fourier Anal. Appl., 26(3):Paper No. 51, 35, 2020.
  * [3] Antoine Ayache and Murad S. Taqqu.  Rate optimality of wavelet series approximations of fractional Brownian motion.  J. Fourier Anal. Appl., 9(5):451–471, 2003.
  * [4] Shuyang Bai and Murad S. Taqqu.  Generalized Hermite processes, discrete chaos and limit theorems.  Stochastic Process. Appl., 124(4):1710–1739, 2014.
  * [5] Karine Bertin, Soledad Torres, and Ciprian A. Tudor.  Maximum-likelihood estimators and random walks in long memory models.  Statistics, 45(4):361–374, 2011.
  * [6] Alexandra Chronopoulou, Ciprian A. Tudor, and Frederi G. Viens.  Self-similarity parameter estimation and reproduction property for non-Gaussian Hermite processes.  Commun. Stoch. Anal., 5(1):161–185, 2011.
  * [7] Alexandra Chronopoulou, Frederi G. Viens, and Ciprian A. Tudor.  Variations and Hurst index estimation for a Rosenblatt process using longer filters.  Electron. J. Stat., 3:1393–1435, 2009.
  * [8] Ingrid Daubechies.  Ten lectures on wavelets, volume 61 of CBMS-NSF Regional Conference Series in Applied Mathematics.  Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992.
  * [9] Lara Daw and Laurent Loosveldt.  Wavelet methods to study the pointwise regularity of the generalized Rosenblatt process.  Electron. J. Probab., 27:1–45, 2022.
  * [10] Roland Lvovich Dobrushin and Peter Major.  Non-central limit theorems for nonlinear functionals of Gaussian fields.  Z. Wahrsch. Verw. Gebiete, 50(1):27–52, 1979.
  * [11] Paul Doukhan, George Oppenheim, and Murad S. Taqqu, editors.  Theory and applications of long-range dependence.  Birkhäuser Boston, Inc., Boston, MA, 2003.
  * [12] Céline Esser and Laurent Loosveldt.  Slow, ordinary and rapid points for Gaussian wavelets series and application to fractional Brownian motions.  ALEA Lat. Am. J. Probab. Math. Stat., 19(2):1471–1495, 2022.
  * [13] Svante Janson.  Gaussian Hilbert spaces, volume 129 of Cambridge Tracts in Mathematics.  Cambridge University Press, Cambridge, 1997.
  * [14] Olav Kallenberg.  On an independence criterion for multiple Wiener integrals.  Ann. Probab., 19(2):483–485, 1991.
  * [15] Andreï Nikolaïevitch Kolmogorov.  Wienersche Spiralen und einige andere interessante Kurven im Hilbertschen Raum.  C. R. (Doklady) Acad. Sci. URSS (N.S.), 26:115–118, 1940.
  * [16] Makoto Maejima and Ciprian A. Tudor.  Wiener integrals with respect to the Hermite process and a non-central limit theorem.  Stoch. Anal. Appl., 25(5):1043–1056, 2007.
  * [17] Makoto Maejima and Ciprian A. Tudor.  Selfsimilar processes with stationary increments in the second Wiener chaos.  Probab. Math. Statist., 32(1):167–186, 2012.
  * [18] Benoit B. Mandelbrot and John W. Van Ness.  Fractional Brownian motions, fractional noises and applications.  SIAM Rev., 10:422–437, 1968.
  * [19] Yves Meyer.  Ondelettes et opérateurs. I.  Actualités Mathématiques. [Current Mathematical Topics]. Hermann, Paris, 1990.  Ondelettes. [Wavelets].
  * [20] Yves Meyer, Fabrice Sellan, and Murad S. Taqqu.  Wavelets, generalized white noise and fractional integration: the synthesis of fractional Brownian motion.  J. Fourier Anal. Appl., 5(5):465–494, 1999.
  * [21] Ivan Nourdin.  Selected aspects of fractional Brownian motion, volume 4 of Bocconi & Springer Series.  Springer, Milan; Bocconi University Press, Milan, 2012.
  * [22] Ivan Nourdin and Giovanni Peccati.  Normal approximations with Malliavin calculus, volume 192 of Cambridge Tracts in Mathematics.  Cambridge University Press, Cambridge, 2012.  From Stein’s method to universality.
  * [23] David Nualart.  The Malliavin calculus and related topics.  Probability and its Applications (New York). Springer-Verlag, Berlin, second edition, 2006.
  * [24] Vladas Pipiras.  Wavelet-type expansion of the Rosenblatt process.  J. Fourier Anal. Appl., 10(6):599–634, 2004.
  * [25] Vladas Pipiras.  Wavelet-based simulation of fractional Brownian motion revisited.  Appl. Comput. Harmon. Anal., 19(1):49–60, 2005.
  * [26] Laurent Schwartz.  Théorie des distributions.  Hermann, 1978.
  * [27] Murad S. Taqqu.  Weak convergence to fractional Brownian motion and to the Rosenblatt process.  Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 31:287–302, 1974/75.
  * [28] Murad S. Taqqu.  Convergence of integrated processes of arbitrary Hermite rank.  Z. Wahrsch. Verw. Gebiete, 50(1):53–83, 1979.
  * [29] Ciprian A. Tudor and Frederi G. Viens.  Variations and estimators for self-similarity parameters via Malliavin calculus.  Ann. Probab., 37(6):2093–2134, 2009.
  * [30] Mark S. Veillette and Murad S. Taqqu.  Properties and numerical evaluation of the Rosenblatt distribution.  Bernoulli, 19(3):982–1005, 2013.