# QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language
Model

Saizhuo Wang  Hang Yuan  Lionel M. Ni  Jian Guo

###### Abstract

Autonomous agents based on Large Language Models (LLMs) that devise plans and
tackle real-world challenges have gained prominence. However, tailoring these
agents for specialized domains like quantitative investment remains a
formidable task. The core challenge involves efficiently building and
integrating a domain-specific knowledge base for the agent’s learning process.
This paper introduces a principled framework to address this challenge,
comprising a two-layer loop. In the inner loop, the agent refines its
responses by drawing from its knowledge base, while in the outer loop, these
responses are tested in real-world scenarios to automatically enhance the
knowledge base with new insights. We demonstrate that our approach enables the
agent to progressively approximate optimal behavior with provable efficiency.
Furthermore, we instantiate this framework through an autonomous agent for
mining trading signals named QuantAgent. Empirical results showcase
QuantAgent’s capability in uncovering viable financial signals and enhancing
the accuracy of financial forecasts.

Machine Learning, ICML

##  1 Introduction

The surge of large language models (LLMs) (OpenAI, 2023; Touvron et al., 2023)
has ignited significant advancements in the realm of autonomous agents (Weng,
2023), expanding their capabilities to plan and solve complex, real-world
tasks. Applying these powerful language models to specialized areas such as
quantitative investment (Wang et al., 2023c) brings its own set of challenges.
One key issue is how to feed these models with specific knowledge from the
field of finance, which would enable them to tackle complex problems like an
expert in that domain.

The primary issue at hand is the source and integration of domain knowledge.
In specialized domains that demand a wealth of experience and technical
acumen, constructing a comprehensive knowledge base often requires intensive
human effort. Meanwhile, traditional methods have relied on incorporating this
knowledge into the agent’s parametric memory through fine-tuning (Hu et al.,
2021) or by referencing external databases during inference via retrieval-
augmented generation techniques (Karpukhin et al., 2020).

However, such approaches face significant hurdles. Crafting an extensive and
accurate domain knowledge base is not only costly in terms of human labor but
may also be impractical or unattainable for certain domains, such as
developing a database of financial alphas (Tulchinsky, 2019) for quantitative
investment strategies. Additionally, there is room in the current research
landscape for a more principled framework that could provide a systematic
assessment of knowledge integration’s impact on the agent, thereby supporting
the advancement of knowledge-enhanced agents.

This paper proposes a principled two-layer framework designed to autonomously
develop a domain-specific knowledge base with minimal human intervention while
maintaining high quality. At its core, the framework operates on a nested loop
system: within the inner loop, the agent refines its responses by interacting
iteratively with a simulated environment defined by an internal knowledge
base. Conversely, in the outer loop, the agent’s responses are evaluated
against the real-world environment, which automatically generates feedback to
further enrich the internal knowledge base. This iterative process propels the
agent towards improved performance, ultimately allowing for the autonomous
accrual of a rich knowledge base.

We bolster the framework with a theoretical analysis, demonstrating that both
the inner and outer loops can efficiently converge towards an optimal
solution. This convergence is substantiated by applying analytical techniques
from reinforcement learning (Liu et al., 2023; Jin et al., 2021), framing the
self-improvement mechanism within the context of a Markov Decision Process.
The resulting framework not only encapsulates a variety of existing self-
improving methods (Wang et al., 2023a; Romera-Paredes et al., 2023) but is
also provably efficient under certain assumptions.

The practical implementation of this framework is realized through QuantAgent,
an autonomous agent tasked with mining financial signals. The agent’s
knowledge base comprises a collection of financial signals, each documented
with its implementation details, the underlying trading idea, performance
metrics, and expert reviews. This setup epitomizes the scenarios where
constructing a robust knowledge base is traditionally challenging, and where
evaluation—backtesting in this context—can be conducted programmatically. Our
empirical findings validate QuantAgent’s capacity for self-improvement, with
the agent successfully generating a comprehensive signal base that enables
more accurate financial forecasting.

The remainder of this paper is organized as follows: we first review related
works in the field, then introduce our framework and its operational pipeline.
Then we present an analysis section discussing the efficiency and practicality
of our algorithm, leading to a presentation of QuantAgent as an instantiation
for financial signal mining. We conclude with empirical results accompanied by
a discussion of broader implications.

##  2 Related Works

###  2.1 LLM-based Autonomous Agents

Recent studies (Weng, 2023; Sumers et al., 2023) have tried to formally
describe the anatomy of an autonomous agent. Despite the differences in terms,
an llm-based autonomous agent basic consists of a set of tools (where the
ability of using tools can be learned (Schick et al., 2023)), a memory (be it
long-term memory such as KB or short-term memory characterized as context), a
planning algorithm (Yao et al., 2022, 2023) that directs what to do. In
practice there have been many applications (Significant Gravitas, 2024;
Nakajima, 2024) that resort to real-world tasks.

###  2.2 Adapting LLM Agents to Domain-specific Tasks

Adapting large language model (LLM)-based autonomous agents to domain-specific
tasks involves techniques such as retrieval-augmented generation (RAG) (Gao et
al., 2024) and fine-tuning (Wang et al., 2023d). RAG enhances LLMs’ ability to
generate more accurate and contextually relevant responses by integrating
external knowledge bases, proving particularly beneficial in specialized
fields requiring extensive factual information (Sun et al., 2023). Fine-
tuning, on the other hand, adjusts pre-trained models on domain-specific
datasets, allowing for improved performance in niche tasks by making the
models more attuned to the unique lexicon and nuances of the domain. These
methodologies facilitate the application of LLM agents across various sectors
by significantly boosting their precision and relevance.

###  2.3 Self-improving LLM agents

Building upon the foundation of adapting LLM agents to domain-specific tasks,
self-improvement emerges as a crucial next step, addressing the challenge of
acquiring domain-specific knowledge, which can be expensive or difficult to
obtain. These self-improvement agents iteratively learn from their environment
through feedback. Recent studies have concentrated on environments including
gaming (Wang et al., 2023a), programming (Haluptzok et al., 2022), and
mathematical problem-solving (Zhu et al., 2023; Romera-Paredes et al., 2023;
Trinh et al., 2024), which naturally offer the continuous feedback necessary
for such learning processes. These environments not only provide rich, dynamic
data for the agents to learn from but also enable the practical application of
self-improvement techniques, reducing the reliance on manually curated,
domain-specific datasets. This approach facilitates the development of LLM
agents that can autonomously enhance their capabilities, adapt to new
challenges, and refine their knowledge base over time, making self-improvement
an integral component of their design.

##  3 Framework

In this section, we elaborate on the two-layered architecture of our proposed
framework. The outer loop, depicted on the left side of Figure 1, represents
the interaction with the real-world environment. Here, the agent’s generated
output is subject to real-world evaluation, and the feedback obtained is
incorporated into the agent’s knowledge base, which in turn informs subsequent
iterations. The right side of Figure 1 details the inner reasoning loop, where
an iterative dialogue between the writer and judge components takes place.
This simulated environment leverages the internal knowledge base and is where
the agent’s reasoning and refinement processes occur. The agent iterates
through this loop, harnessing the shared context buffer for knowledge
retrieval and inference until a satisfactory solution is produced or a pre-set
threshold is reached.

Figure 1: Our proposed framework. Left: The outer feedback loop. The agent
generates an answer to the problem, submit to the real environment for
evaluation, and receives feedback. The feedback is updated to the knowledge
base (KB) of the agent that serves further usage. Right: The pipeline of the
inner reasoning loop. During each loop iteration, the following things happen
sequentially: ① The writer retrieves relevant knowledge from the KB to inform
the initial answer; ② The writer generates an answer utilizing the retrieved
knowledge; ③ The judge retrieves relevant knowledge from KB to hint its
review; ④ The answer undergoes review, receiving a score and advice for
improvement, which is then used for the next iteration. Both the writer and
the judge are LLMs with specifically designed prompts and each step in the
iteration uses buffer contents as context. This iterative process stops when
the maximum number of steps ($T$) is reached or the score is high enough.
Algorithm 1 The inner reasoning loop

0: The problem $p$, Knowledge base $\phi$, Reward threshold $\beta$, Max
iteration limit $T$.

0: An answer $\alpha$

1: Judge $\leftarrow$ LLM prompted with judge prompt

2: Agent $\leftarrow$ LLM prompted with agent prompt

3: ctx $\leftarrow$ {$p$}

4: $t\leftarrow 0$

5: repeat

6: $K_{1}$ $\leftarrow$ retrieve(Agent, ctx, $\phi$)

7: ctx $\leftarrow$ ctx $\cup$ {$K_{1}$}

8: $\hat{\alpha}\leftarrow$ Agent.generate(ctx)

9: $K_{2}\leftarrow$ retrieve(Judge, ctx, $\phi$)

10: ctx $\leftarrow$ ctx $\cup$ {$\hat{\alpha},K_{2}$}

11: score, review $\leftarrow$ Judge.eval($\hat{\alpha}$, ctx)

12: $t\leftarrow t+1$

13: ctx $\leftarrow$ ctx $\cup$ {review}

14: until $r\geq\beta$ or $t\geq T$

15: return $\hat{\alpha}$

###  3.1 The Inner Reasoning Loop

The inner loop functions as a simulated reasoning environment where an LLM or
rule-based system interacts with a knowledge repository. It employs a memory
buffer that initially contains the user’s query and is enriched iteratively
with data from the knowledge base. The agent poses queries to the knowledge
base, which in turn provides pertinent information. A judge, which could be a
rule set or an LLM, evaluates this information to ensure it satisfies the
user’s query, concluding the loop when a response is formulated or a
predefined condition is met.

####  3.1.1 Components

##### Knowledge Base

The knowledge base acts as a repository containing records of the agent’s
prior outputs, associated performance scores, and feedback. The agent
formulates queries to extract necessary information, which are then processed
into embeddings for efficient retrieval of relevant records, aiming to
optimize the trade-off between performance accuracy and response variety.

##### Context Buffer

The context buffer maintains a record of the ongoing interaction, holding all
previous exchanges and information. This cumulative record ensures consistency
and coherence in the agent’s reasoning process, allowing past knowledge to
inform future responses.

##### Writer

The writer is responsible for constructing responses based on the data
retrieved from the knowledge base. It is designed to incrementally refine its
outputs, integrating feedback from the judge to enhance the quality of
subsequent responses.

##### Judge

The judge serves as an evaluator, providing feedback by scoring the writer’s
outputs. Its effectiveness as an assessor is contingent on the quality of the
knowledge base, and it plays a critical role in calibrating the agent’s
outputs to improve accuracy over time.

####  3.1.2 Procedure

##### One Iteration

Within a single iteration, the writer commences by sourcing relevant knowledge
from the knowledge base. Utilizing this knowledge, the writer formulates a
response. This response is then assessed by the judge, who provides a score
and feedback, which is incorporated back into the context buffer to refine the
writer’s next response.

##### Iterative Process

The iterative process is predicated on the accumulation of information leading
to progressively improved responses. The loop is repeated, leveraging feedback
to incrementally enhance the quality of responses, until a predefined
performance threshold is achieved or an optimal response is determined.

Remarks: The guiding principle behind the inner loop is that through a robust
mechanism for generating responses, the writer will, with sufficient
iterations, accumulate adequate information from the knowledge base to
consistently satisfy the judge’s criteria. This iterative enrichment is
expected to guide the writer towards producing the optimal answer. A formal
analysis of this convergence process is presented in Sec. 4.1.2.

Algorithm 2 The Outer Feedback Loop

0: Problem description $p$, Other inner loop params $c$, Number of outer loop
iterations $K$

0: Optimized agent knowledge base $\phi$

1: $\phi\leftarrow\emptyset$

2: for $i=1$ to $K$ do

3: $a$ $\leftarrow$ innerLoop($p$, $\phi$, $c$)

4: $f\leftarrow$ Real-world feedback for $a$ given problem $p$

5: Update $\phi$ with $a$ and $f$

6: end for

7: return $\phi$

###  3.2 The Outer Feedback Loop

The outer loop encapsulates the agent’s iterative interactions with the real-
world environment, where its generated outputs are evaluated and refined.

##### Environment Feedback

The environment provides feedback in the form of performance scores and
qualitative reviews, which may be generated by a sophisticated LLM. This
feedback serves to inject new insights into the agent’s decision-making
process, with the potential to enhance future performances.

##### Knowledge Update

Following the receipt of feedback, the knowledge base undergoes an update
process. This process incorporates sanity checks to ensure the integrity and
relevance of the new information. The rules for updating are crafted to
maintain a comprehensive database that includes a spectrum of experiences,
both successful and otherwise. This approach ensures a diverse learning
context for the agent, promoting a nuanced understanding and adaptation to the
real-world scenarios.

Remarks: A distinct contrast exists between the judge’s role in the inner loop
and the environment’s feedback in the outer loop. The former can be
characterized as providing rapid, cost-effective, albeit less precise
evaluations based on a limited knowledge set. In contrast, the latter is akin
to a standard of truth, often more resource-intensive but offering a higher
fidelity of assessment. As the volume of iterations within the outer loop
increases, the inner loop judge accumulates extensive real-world experiences,
which gradually refines its capacity for delivering high-fidelity evaluations
and feedback.

###  3.3 Comparison with existing methods

Many existing methods can be regarded as specific implementations of our
framework. On the one hand, if we discard the outer loop, then the family of
self-refinement methods (Madaan et al., 2023) can be regarded, with two LLMs
acting as actor and critic either with or without a knowledge base. On the
other hand, if we reduce the inner loop to a vanilla retrieval-augmented
generation procedure, then many self-improving methods such as Voyager (Wang
et al., 2023a) and FunSearch (Romera-Paredes et al., 2023) can be implemented.

##  4 Analysis

In this section we will analyze the efficiency (i.e. the agent algorithm can
converge to a optimal solution to this problem asymptotically), and cost (in
terms of both token cost and inference time cost). The motivation of our
analysis is 1). understand the effect of each design component in our system
2). analyze whether it is practical to deploy it to real world.

###  4.1 Efficiency

We prove the efficiency of our agent algorithm at two levels: the inner loop
level and the outer loop level. Efficiency at the inner loop indicates that
the agent can reach the best possible answer to the problem given current
knowledge base, and efficiency in the outer loop indicate that the performance
gap between the optimal policy in simulated environment characterized by the
KB and that in the real world can converge as the outer loop iteration
increases. The two efficiency guarantees together forms the guarantee that the
policy can converge to global optimal, indicating the efficiency of our agent
algorithm. To analyze the efficiency, we first introduce the formulation of
our problem.

Figure 2: The analysis framework. Left: An abstract view of the inner and
outer loop, where three different environments (real environment $\theta$,
simulated environment $\bar{\theta}$ and the LLM-inferred environment
$\hat{\theta}$) are characterized. Right: The chain of proofs that links the
agent’s actual policy $\hat{\pi}$ and the optimal policy in the real
environment $\pi^{*}$

####  4.1.1 Formulation

We formulate the process of generating an answer to the problem (essentially
the inner loop) as a Markov Decision Process (MDP). This MDP is defined by the
tuple $(\mathcal{S},\mathcal{A},T,r,\gamma)$, where each component is
characterized as follows:

  * •

State Space $\mathcal{S}$: The state at any time $t$, denoted as $s_{t}$, is a
composite of the original problem $p$, and all the information contained in
the shared context buffer. Initially $s^{p}_{t_{0}}$ is an empty set,
indicating the start of the reasoning process. $s_{t}$ that encapsulates all
historical information explored up to time $t$, and we call it the information
state (Liu et al., 2023).

  * •

Action Space $\mathcal{A}$: Action is w.r.t. the writer since only for writer
action is meaningful. Action is like generating an answer to the problem,
based on current information at hand. action can also be knowledge base
queries since it also changes the information state.

  * •

Transition Function $T$: Defined as $T(s^{\prime}|s,a)$, states that based on
current information at hand, what will the information state change given an
action. This has several implementations: 1). KB responding to a query action
2). Judge reviewing an answer action. So both the KB and the judge constitutes
the transition function.

  * •

Reward Function $r$: This function, $r(s_{t})$, assigns a value to the
information state $s_{t}$. $r$ can be either a continuous value, indicating
the absolute score of an answer, or a $0/1$ value indicating whether the
answer is sufficient for this problem.

  * •

Discount Factor $\gamma$: It defines the weight of future rewards in the
agent’s consideration, shaping the strategic depth of the reasoning process by
emphasizing the importance of long-term outcomes. In practice, we use
$\gamma\in(0,1)$ to ensure that the value function $V^{\pi}_{\theta}(s)$ is
bounded, which is important for analysis.

Given a policy $\pi:\mathcal{S}\mapsto\mathcal{A}$ and an environment
parameterized by $\theta$, we define its value function and Q-function as
follows:

$V^{\pi}_{\theta}(s)=\mathbb{E}\bigl{[}\sum_{t=0}^{\infty}\gamma^{t}r_{\theta}(s_{t},a_{t})|s_{0}=s\bigr{]}$
(1)
$Q^{\pi}_{\theta}(s,a)=\mathbb{E}\bigl{[}\sum_{t=0}^{\infty}\gamma^{t}r_{\theta}(s_{t},a_{t})|s_{0}=s,a_{0}=a\bigr{]},$
(2)

where the expectation is taken over $a_{t}\sim\pi(s_{t})$ and $s_{t+1}\sim
T_{\theta}(\cdot|s_{t},a_{t})$ for all $t>0$. Here $\theta$ can be considered
as knowledge base contents. Specifically, $\theta$ affects the transition
function $T_{\theta}(\cdot|s_{t},a_{t})$ by determining on the query results
of a given query action, and the reward function $r_{\theta}(s)$ is also
affected by the knowledge base since the judge depend on it. The goal of the
inner loop, therefore, is to learn a policy that maximizes
$V^{\pi}_{\theta}(s)$ for all $s\in\mathcal{S}$ so as to find enough
information for arbitrary user questions.

Our goal is to prove that our agent is provably efficient in both the inner
and outer loop. Namely, the Bayesian regret of the agent

$\mathcal{R}(TK)=\mathbb{E}\bigl{[}\sum_{k=1}^{K}\sum_{t^{k}=1}^{T}V^{\pi^{*}}_{\theta}(s_{t})-V^{\hat{\pi}_{t^{k}}}_{\theta}(s_{t})\bigr{]},$
(3)

is sublinear in $KT$.

####  4.1.2 The inner loop

Following (Liu et al., 2023), we make the following assumptions on the inner
loop:

######  Assumption 4.1.

During the in-context inference step, the LLM writer implicitly performance
Bayesian inference of the simulated environment’s parameter.

Assumption 4.1 essentially states that LLM performs implicit Bayesian
inference $p(\hat{\theta}|s)$ of the environment parameter $\hat{\theta}$
given the information state $s$ and its prior knowledge $p(\hat{\theta})$
obtained via pretraining. This mechanism has also been verified in previous
works (Xie et al., 2021). Based on the estimated environment parameter, we
employ a planning mechanism to ensure optimality on $\hat{\theta}$:

######  Definition 4.2.

($\epsilon$-Optimality (Liu et al., 2023)) Under an environment parametrized
by $\theta$, a policy $\pi$ satisfies $\epsilon$-optimality if the following
condition holds with high probability

$\max_{s}V^{\text{PL}^{*}(\theta)}_{\theta}(s)-V^{\pi}_{\theta}(s)\leq\epsilon,$
(4)

In this way, provable efficiency in the inner loop states that by iteratively
refining its answer under the growing information and judge’s feedback,

######  Lemma 4.3.

The Bayesian regret of the planning agent in the inner loop is sublinear in
the number of inner loop iterations $T$

###### Proof.

Detailed proof can be seen in (Liu et al., 2023; Wang et al., 2023b) ∎

The intuition behind this proof is that since LLM inference is implicit
Bayesian inference, its posterior estimation of the simulated world parameter
gets more and more accurate, with information gap eventually converges. Given
the optimality of the planning algorithm, the sub-optimality depends only on
the model estimation error. So as information converges, the model esitmation
also converges, leading to optimal policy in the end.

Meanwhile, it is also notable that one-shot method which correspond to the
close-loop solution has no theoretical guarantee of efficiency, meaning that
they are not theoretically convergent. This is because the information in LLM
inference does not accumulates as iteration grows. However, in other works
such method also works quite well, indicating the actual information gap after
1 shot may not be that large. Moreover, other works also do not explicitly
incorporate a planning mechanism while also achieving good results, and this
might be because the sub-optimality might not be that severe either in real-
world practice.

####  4.1.3 The outer loop

Efficiency in the outer loop essentially is the result from pessimism. We
first assume that the optimal policy on the knowledge base can be trained via
pessimism. In this way, the knowledge base can be regarded as a offline
dataset and obtaining such an optimal policy can be regarded as performing
offline RL on this dataset. (Jin et al., 2021) states that the performance gap
between the best policy learned offline and the best policy online can be
bounded by the information gap, if the offline policy is learnt with
pessimism. Our analysis will assume this and based on this to get our
efficiency proof.

######  Assumption 4.4.

Given a simulated environment parametrized by $\bar{\theta}$ characterized by
an offline knowledge base, the optimal policy $\bar{\pi}$ can be obtained
pessimistic value iteration (PEVI) (Jin et al., 2021).

######  Lemma 4.5.

(Efficiency of Pessimism) Under Assumption 4.4, the performance gap between
$\bar{\pi}$ and $\pi^{*}$ in the real environment is bounded by the intrinsic
uncertainty caused by insufficient coverage of the knowledge base.

###### Proof.

The detailed proof can be found in Section 4 of (Jin et al., 2021). ∎

Lemma 4.5 guarantees that under pessimism, the sub-optimality is only related
to the estiamtion inaccuracy in model parameters between the real environment
and the simulated one characterized by the knowledge base. And as the KB
accumulates more and more information about the real environment, making the
simulated one closer to the real one, the optimal policy trained using
pessimism on this simulated environment should also have a converging
performance gap. This theoreical results bridges the optimal policy in the
real env and simu env, which together with the inner loop efficiency links
together the gap between the policy learned by the agent and the optimal one
in the real env.

####  4.1.4 Overall result

Combining provable efficiency in the inner and outer loop, we can bridge the
overall performance gap

######  Theorem 4.6.

The Bayesian regret of the LLM agent in the real environment,
$\mathcal{R}(TK)$ in Eq. 3, is sublinear in KT.

###### Proof.

We give a sketch of the proof here. Detailed proof of each term can be found
in (Liu et al., 2023; Jin et al., 2021). The performance gap can be
essentially decomposed as follows:

$\displaystyle V^{\pi^{*}}_{\theta}(s_{t})-V^{\hat{\pi}}_{\theta}(s_{t})$
$\displaystyle=\underbrace{V^{\pi^{*}}_{\theta}(s_{t})-V^{\bar{\pi}}_{\theta}(s_{t})}_{\text{Term
A}}$
$\displaystyle\quad+\underbrace{V^{\bar{\pi}}_{\theta}(s_{t})-V^{\bar{\pi}}_{\mathclap{\bar{\theta}}}(s_{t})}_{\text{Term
B}}$
$\displaystyle+\underbrace{V^{\bar{\pi}}_{\mathclap{\bar{\theta}}}(s_{t})-V^{\hat{\pi}}_{\mathclap{\bar{\theta}}}(s_{t})}_{\text{Term
C}}+\underbrace{V^{\hat{\pi}}_{\mathclap{\bar{\theta}}}(s_{t})-V^{\hat{\pi}}_{\theta}(s_{t})}_{\text{Term
D}}$ (5)

When taking sum over $T$ and $K$, Term A, B, given pessimism assumption
(Assumption 4.4) can be bounded by the intrinsic uncertainty in the offline
dataset (lemma 4.5), which is sublinear in $K$ under modest assumptions (e.g.
linear MDP) of the underlying MDP. Similarly, for Term D, it is also bounded
by the information gap which is sublinear in $K$ in the same case. For Term C,
according to Lemma 4.3, it is sublinear in $T$. Hence, the LHS, when taken sum
in Eq. 3, is sublinear in $KT$. ∎

###  4.2 Cost Analysis

##### Token Cost

In the self-improvement phase, the inner loop incurs a token cost that
escalates with the square of the planning horizon $H$ and the number of close-
loop interaction rounds $T$. This yields a computational complexity of
$O(T^{2}H)$ for a single outer loop iteration. When considering multiple
rounds of outer loop iterations, necessary for the agent’s self-improvement,
the cumulative token cost across $K$ such iterations scales to $O(KT^{2}H)$.
During the inference phase, where the outer loop is not executed, the token
cost remains at $O(T^{2}H)$, assuming that the complexity of generating
responses does not change significantly from training to inference.

##### Time Cost

The time cost for each iteration within the outer loop is influenced by both
the planning horizon and a constant factor representing the baseline
computational overhead. Assuming the computational time for key operations is
constant and each value computation incurs one unit of time, a single outer
loop iteration requires $T(H+\text{Const})$ units of time. Consequently, the
entire training phase, encompassing $K$ iterations of the outer loop, demands
$O(KTH)$ time. In the inference phase, devoid of the outer loop, the time
complexity is reduced to $O(HT)$.

##  5 Experiment

In this section we introduce the experiment settings in our paper. Our
experiment is conducted on a domain-specific scenario, so we will introduce
tasks background, and then the most important setups in our experiment. For
detailed information see appendix.

###  5.1 Background

Our task is financial signal mining. Financial signals are predictive signals
computed from financial market data that can be used for financial prediction.
Designing financial signals requires good market understanding, code
implementation skills, and mathematical skills such as numerical analysis. The
primary goal of this task is to get good financial signals that achieves high
predictive power. Given a trading idea, we would also like the agent to
improve its We would also like to get a diversified set of signals on which we
can build machine learning models to combine them to get better predictions.

###  5.2 Problem Setup

The general goal is to generate good financial signals evaluated by our
measurements. To achieve this goal, at each outer loop iteration, the agent is
given a trading idea, and then asked to generate financial signals that both
reflects this trading idea while also achieves good performance (by taking
other considerations). The trading idea is sampled from a distribution, in our
experiment generated by another LLM. The implementation of a financial signal
is essentially a piece of code (or equivalently a function) with pre-defined
schema. Similar to FunSearch, we define a template shown in the appendix and
let the LLM agent implement the function codes for that alpha.

##### Dataset

We selected a universe of 500 stocks on Chinese A-share market. The time range
of these data is taken in year 2023. The basic market data used for computing
financial signals contains the volume and price day collect at market close at
each day.

##### Foundation LLM

We selected gpt-4-0125-preview as the version for our foundation model.

###  5.3 Evaluation protocol

##### Predictive Performance

To assess the predictive power of the financial signals, we calculate the
Information Coefficient (IC) with respect to future returns. IC is the Pearson
correlation computed for each cross-section (all stocks at a single time
point) and then averaged across all time points. To determine the efficacy of
the Knowledge Base (KB) as a foundational dataset for signal generation, we
examine the IC of signals produced by a machine learning model, in this case,
XGBoost regression trees, using the KB as the feature input. This evaluates
the KB’s potential as a reliable source for creating predictive models in
finance. Additionally, we analyze the signal’s Sharpe ratio to evaluate its
ability to generate satisfactory investment returns.

##### Signal Quality

For numerical quality, we look at the number of valid and unique entities the
signals pertain to, ensuring they are capable of differentiating stocks. In
terms of trading idea similarity, we measure how well the signals correspond
to their underlying trading ideas. This relevance is critical for the
potential use of LLMs as investment research assistants. To quantify this, we
conduct pairwise comparisons where an LLM evaluates two signals generated from
the same trading idea by different agents. The LLM selects the more accurate
signal, and from these pairwise comparisons, we construct a win-rate matrix.
The aggregated results from this matrix form a leaderboard that ranks the
agents on their ability to produce signals that are true to the original
trading concepts. This ranking offers insights into the proficiency of
different agents in capturing the essence of trading ideas within their signal
implementations.

##  6 Results

This section presents experimental results corroborating our theoretical
findings. We highlight the main outcomes indicative of the agent’s self-
improvement capabilities and analyze the effect of inner and outer loop via
different experiments.

###  6.1 Evidence of Self-Improvement

Figure 5 shows the consistence performance improvement in terms of both
training on cumulative signals and training on individual segments of alphas.
The increasing predictive accuracy shows that, as the agent self-improves, it
can produce stronger signals for better predictive power. Note that

Figure 3: Evolving curve of averaged grouped single-alpha performance. From
left to right: entry valid ratio, IC score, return, Sharpe ratio Figure 4:
Trading idea relevance comparison. From left to right: with inner and outer
loop; without inner loop, only outer loop;without outer loop, only inner loop;
no inner or outer loop. Figure 5: Increasing predictive accuracy in the
trained model as the number of alpha accumulates. Y-axis indicates the MSE
error

###  6.2 Improving Alpha Relevance

To measure signal quality, we sorted the signals generated throughout the
whole process according to their generated iterations, and segment them into
equally-numbered groups for comparison in terms of accurately conveying the
underlying trading idea, judge by GPT-4. Figure 4 shows the distribution of
the win-rate. The pattern of accumulating winning rates to the upper right
corner indicates that as the model evolves, the agent gains better skill in
writing good-quality signals, verifying the effectiveness of self-improving.
Notably, the diminishing pattern in the rightmost matrix demonstrate the
effectiveness of both our inner and outer loop.

###  6.3 Effect of the Inner and Outer Loop

Figure 3 demonstrates the evolution of single-alpha attributes across the
whole process. While it is not apparent to see significant performance
differences, the trend of the blue curve demonstrates the effectivenss of
self-improvement in all metrics.

##  7 Discussion

Our research illustrates the efficacy of autonomous agents, particularly in
quantitative investment, showcasing their capability for self-improvement and
adaptation through simulated environments. This advancement not only
highlights their potential as tools for financial analysis and risk management
but also opens the door to their application across various complex domains
beyond finance, such as healthcare and logistics, by customizing the knowledge
base and feedback mechanisms. Despite these promising developments, challenges
such as the dependency on knowledge base quality and the need for
computational optimization remain. Future endeavors will aim to enhance the
agents’ learning efficiency, diversify their application scenarios, and
explore real-time adaptation to dynamic environments. Ultimately, these
efforts underscore the vast potential of LLM-based agents to revolutionize
decision-making processes across industries, marking a significant stride
towards the era of intelligent automation.

##  8 Broader impact

This paper presents work with the goal of advancing the research direction of
building self-improving autonomous LLM agents. There are many potential
societal consequences of our work, none of which we feel must be specifically
highlighted here.

## References

  * Gao et al. (2024) Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Guo, Q., Wang, M., and Wang, H.  Retrieval-Augmented Generation for Large Language Models: A Survey, January 2024.  URL http://arxiv.org/abs/2312.10997.  arXiv:2312.10997 [cs].
  * Haluptzok et al. (2022) Haluptzok, P., Bowers, M., and Kalai, A. T.  Language Models Can Teach Themselves to Program Better.  September 2022.  URL https://openreview.net/forum?id=SaRj2ka1XZ3.
  * Hu et al. (2021) Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W.  LoRA: Low-Rank Adaptation of Large Language Models, October 2021.  URL http://arxiv.org/abs/2106.09685.  arXiv:2106.09685 [cs].
  * Jin et al. (2021) Jin, Y., Yang, Z., and Wang, Z.  Is Pessimism Provably Efficient for Offline RL?  In _Proceedings of the 38th International Conference on Machine Learning_ , pp. 5084–5096. PMLR, July 2021.  URL https://proceedings.mlr.press/v139/jin21e.html.  ISSN: 2640-3498.
  * Karpukhin et al. (2020) Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t.  Dense Passage Retrieval for Open-Domain Question Answering, September 2020.  URL http://arxiv.org/abs/2004.04906.  arXiv:2004.04906 [cs].
  * Liu et al. (2023) Liu, Z., Hu, H., Zhang, S., Guo, H., Ke, S., Liu, B., and Wang, Z.  Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency, October 2023.  URL http://arxiv.org/abs/2309.17382.  arXiv:2309.17382 [cs].
  * Madaan et al. (2023) Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Welleck, S., Majumder, B. P., Gupta, S., Yazdanbakhsh, A., and Clark, P.  Self-Refine: Iterative Refinement with Self-Feedback.  _CoRR_ , abs/2303.17651, 2023.  doi: 10.48550/ARXIV.2303.17651.  URL https://doi.org/10.48550/arXiv.2303.17651.  arXiv: 2303.17651.
  * Nakajima (2024) Nakajima, Y.  yoheinakajima/babyagi, February 2024.  URL https://github.com/yoheinakajima/babyagi.  original-date: 2023-04-03T00:40:27Z.
  * OpenAI (2023) OpenAI.  GPT-4 Technical Report.  _CoRR_ , abs/2303.08774, 2023.  doi: 10.48550/ARXIV.2303.08774.  URL https://doi.org/10.48550/arXiv.2303.08774.  arXiv: 2303.08774.
  * Romera-Paredes et al. (2023) Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., Kohli, P., and Fawzi, A.  Mathematical discoveries from program search with large language models.  _Nature_ , pp. 1–3, December 2023.  ISSN 1476-4687.  doi: 10.1038/s41586-023-06924-6.  URL https://www.nature.com/articles/s41586-023-06924-6.  Publisher: Nature Publishing Group.
  * Schick et al. (2023) Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T.  Toolformer: Language Models Can Teach Themselves to Use Tools.  November 2023.  URL https://openreview.net/forum?id=Yacmpz84TH.
  * Significant Gravitas (2024) Significant Gravitas.  AutoGPT, February 2024.  URL https://github.com/Significant-Gravitas/AutoGPT.  original-date: 2023-03-16T09:21:07Z.
  * Sumers et al. (2023) Sumers, T. R., Yao, S., Narasimhan, K., and Griffiths, T. L.  Cognitive Architectures for Language Agents, September 2023.  URL http://arxiv.org/abs/2309.02427.  arXiv:2309.02427 [cs].
  * Sun et al. (2023) Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y., Ni, L., Shum, H.-Y., and Guo, J.  Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph.  October 2023.  URL https://openreview.net/forum?id=nnVO1PvbTv.
  * Touvron et al. (2023) Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hosseini, S., Hou, R., Inan, H., Kardas, M., Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A., Koura, P. S., Lachaux, M.-A., Lavril, T., Lee, J., Liskovich, D., Lu, Y., Mao, Y., Martinet, X., Mihaylov, T., Mishra, P., Molybog, I., Nie, Y., Poulton, A., Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith, E. M., Subramanian, R., Tan, X. E., Tang, B., Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and Scialom, T.  Llama 2: Open Foundation and Fine-Tuned Chat Models, July 2023\.  URL http://arxiv.org/abs/2307.09288.  arXiv:2307.09288 [cs].
  * Trinh et al. (2024) Trinh, T. H., Wu, Y., Le, Q. V., He, H., and Luong, T.  Solving olympiad geometry without human demonstrations.  _Nature_ , 625(7995):476–482, January 2024.  ISSN 1476-4687.  doi: 10.1038/s41586-023-06747-5.  URL https://doi.org/10.1038/s41586-023-06747-5.
  * Tulchinsky (2019) Tulchinsky, I.  Introduction to Alpha Design.  In _Finding Alphas_ , pp. 1–6. John Wiley & Sons, Ltd, 2019\.  ISBN 978-1-119-57127-8.  doi: https://doi.org/10.1002/9781119571278.ch1.  URL https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119571278.ch1.
  * Wang et al. (2023a) Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A.  Voyager: An Open-Ended Embodied Agent with Large Language Models, October 2023a.  URL http://arxiv.org/abs/2305.16291.  arXiv:2305.16291 [cs].
  * Wang et al. (2023b) Wang, S., Liu, Z., Wang, Z., and Guo, J.  A Principled Framework for Knowledge-enhanced Large Language Model, November 2023b.  URL http://arxiv.org/abs/2311.11135.  arXiv:2311.11135 [cs].
  * Wang et al. (2023c) Wang, S., Yuan, H., Zhou, L., Ni, L. M., Shum, H.-Y., and Guo, J.  Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment, July 2023c.  URL http://arxiv.org/abs/2308.00016.  arXiv:2308.00016 [cs, q-fin].
  * Wang et al. (2023d) Wang, Y., Zhong, W., Li, L., Mi, F., Zeng, X., Huang, W., Shang, L., Jiang, X., and Liu, Q.  Aligning Large Language Models with Human: A Survey, July 2023d.  URL http://arxiv.org/abs/2307.12966.  arXiv:2307.12966 [cs].
  * Weng (2023) Weng, L.  LLM Powered Autonomous Agents, June 2023.  URL https://lilianweng.github.io/posts/2023-06-23-agent/.  Section: posts.
  * Xie et al. (2021) Xie, S. M., Raghunathan, A., Liang, P., and Ma, T.  An Explanation of In-context Learning as Implicit Bayesian Inference.  October 2021.  URL https://openreview.net/forum?id=RdJVFCHjUMI.
  * Yao et al. (2022) Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y.  ReAct: Synergizing Reasoning and Acting in Language Models.  September 2022.  URL https://openreview.net/forum?id=WE_vluYUL-X.
  * Yao et al. (2023) Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K. R.  Tree of Thoughts: Deliberate Problem Solving with Large Language Models.  November 2023.  URL https://openreview.net/forum?id=5Xc1ecxO1h.
  * Zhu et al. (2023) Zhu, Z., Xue, Y., Chen, X., Zhou, D., Tang, J., Schuurmans, D., and Dai, H.  Large Language Models can Learn Rules, October 2023.  URL http://arxiv.org/abs/2310.07064.  arXiv:2310.07064 [cs].

##  Appendix A Detailed Problem Backgrounds

By discerning the patterns of market changes to write trading signals that can
capture and profit from these patterns, in the field of quantitative finance,
we typically refer to such trading signals or characteristics that have
predictive power over future markets as alphas. Usually, one writes a program
to calculate historical market data to implement the core logic and ideas
behind this trading signal, and then validates through calculation and
backtesting whether the trading signal can be profitable. Moreover, a single
alpha signal is unlikely to cover all market characteristics. To more
accurately predict future markets, a set of trading signals with low
correlation to each other, covering various market dimensions, is needed.
Combining these signals through model construction allows for an accurate
market forecast.

Therefore, an excellent quantitative finance researcher needs to be able to
translate their market insights effectively into a program that constructs
these trading signals. There are two challenges here: on one hand, how to
translate your observations and understanding of the market, which are often
described in natural language, into a computer program accurately and
fittingly; on the other hand, how to build a high-quality, predictive trading
signal, which may involve skills and experience in constructing quality
trading signals and programs, numerical processing, and refinement. You might
need to try combining other signals with yours to make it more effective at
predicting the market and profiting.

A human researcher often requires many years of experience and a high level of
talent to perform this job well, and the efficiency is not high—it is
difficult to rapidly build a rich set of alphas to enhance market prediction
in the short term. A rich and diverse library of alphas with strong predictive
capabilities usually requires years of accumulation.

After LLMs demonstrated their understanding of natural language and ability to
write programs, some works, such as Alpha-GPT, have attempted to use large
models to generate trading signal programs by understanding natural language
trading ideas. However, there are technical challenges involved, among which
the most significant is that to generate better signals that more closely
match the trading ideas, the large model needs knowledge enhancement.

###  A.1 An example alpha

⬇

1from idea_factor import Factor

2import pandas as pd

3import numpy as np

4

5class VolatilityBreakoutSignal(Factor):

6 def __init__(self):

7 self.name = "VolatilityBreakoutSignal"

8 self.window_length = 14

9 self.inputs = ["high", "low", "pre_close"]

10

11 def calc(self, data):

12 # Calculate the components of the True Range (TR)

13 tr = np.maximum(data[’high’] - data[’low’],

14 np.maximum(abs(data[’high’] - data[’pre_close’].shift(1)),

15 abs(data[’low’] - data[’pre_close’].shift(1))))

16

17 # Calculate the Average True Range (ATR) using a rolling mean of the True
Range

18 atr = tr.rolling(window=self.window_length).mean()

19

20 # Define breakout threshold as 1.5 times the ATR from the previous day’s
high

21 breakout_threshold = data[’high’].shift(1) + 1.5 * atr

22

23 # Generate the signal indicating a volatility breakout

24 signal = (data[’high’] > breakout_threshold) * 1

25

26 # Normalize the signal using the distance from the breakout threshold
divided by ATR

27 # to make it proportional to the size of the breakout

28 normalized_signal = ((data[’high’] - breakout_threshold) /
atr).where(signal == 1, 0)

29

30 # Ensure all values are non-negative, and fill any NaN values with 0

31 normalized_signal = normalized_signal.clip(lower=0).fillna(0)

32

33 return normalized_signal

##  Appendix B Detailed Experimental Settings

###  B.1 The financial signal mining runtime environment

We use idea-factor a framework that we developed to define trading signal /
alpha

In the idea-factor framework, trading signals are implemented by defining a
new class that inherits from a base class named Factor. This base class
handles technical aspects of signal calculation and maintenance.

For a researcher to develop a trading signal, all you should care about is
following:

1\. Defining Member Variables: The class must have three member variables: \-
name: The signal’s name, which should also be reflected in the class name. \-
window-length: The historical data window length needed for each signal
calculation. \- inputs: Specifies the base data required for the signal
calculation.

2\. Implementing the calc Function: This is where the trading signal’s logic
is realized. The framework uses a loop to call this function, allowing the
calculation of the trading signal for historical periods. You can use various
Python computational libraries, like scipy, to enhance your calculations.
Additionally, you can define multiple auxiliary member functions within the
class for complex calculations, which can be utilized within the calc
function. \- Input Parameters : a dictionary called data is passed as an input
parameter, containing base data mapped from inputs. Each key corresponds to an
input variable, and the value is a pandas.DataFrame with rows as dates and
columns as stocks. \- Return Value: The function should return a
pandas.DataFrame, with columns representing the universe of stocks and rows
indexed by dates. The idea-factor framework will then use the last row of this
DataFrame as the signal values for the current period.

###  B.2 Base data

In this experiment, we provide the following data for creating a trading
signals.

  * •

close: Closing Price

  * •

high: Highest Price

  * •

low: Lowest Price

  * •

open: Opening Price

  * •

pre-close: Previous Day’s Closing Price

  * •

return: The rate of change of today’s closing price relative to the previous
day’s closing price

  * •

shares: Total Shares Outstanding

  * •

tradenum: Number of Trades

  * •

turnover: Turnover Rate

  * •

value: Transaction Amount

  * •

volume: Trading Volume

  * •

vwap: Volume-Weighted Average Price

##  Appendix C Detailed Experimental Results

###  C.1 A self-improving example of inner cycle

Trading Idea: I observed that three consecutive candlesticks each closing
higher than the preceding one. This pattern suggests strong buying pressure in
the market. Please implement a trading signal based on this observation

Trading Signal implementation 1st version:

⬇

1class ThreeSoldierSignal(Factor):

2 def __init__(self):

3 self.name = "Three Advancing White Soldiers"

4 # Assuming a 3-day pattern, extend window to capture necessary data for
calculation

5 self.window_length = 4 # Extra day to account for comparison with previous
conditions

6 self.inputs = ["close", "open"]

7

8 def _check_pattern(self, opens, closes):

9 """

10 Check if the last three days form the (Three Advancing White Soldiers)
pattern.

11

12 """

13 # Criteria for this pattern:

14 # 1. Three consecutive days with higher closes than opens

15 higher_close_than_open = (closes.shift(-1) > opens.shift(-1)) &
(closes.shift(-2) > opens.shift(-2)) & (closes > opens)

16

17 # 2. Each day closes higher than the previous day

18 consecutive_higher_closes = (closes > closes.shift(-1)) & (closes.shift(-1)
> closes.shift(-2))

19

20 # 3. The opening of each day is within the body of the previous day

21 # Ignoring the gap between days as it may not be strictly applicable in all
markets

22 open_within_previous_body = (opens.shift(-1) > opens.shift(-2)) &
(opens.shift(-1) < closes.shift(-2)) & \

23 (opens > opens.shift(-1)) & (opens < closes.shift(-1))

24

25 # Combine conditions, if all true pattern detected

26 if higher_close_than_open.iloc[-1] and consecutive_higher_closes.iloc[-1]
and open_within_previous_body.iloc[-1]:

27 return 1

28 else:

29 return 0

30

31 def calc(self, data):

32 # Initialize signal DataFrame

33 signal = pd.DataFrame(index=data[’close’].index,
columns=data[’close’].columns)

34

35 # Iterate over columns (stocks) and calculate signal

36 for stock in data[’close’].columns:

37 combined_signal =
data[’open’][stock].rolling(window=self.window_length).apply(

38 lambda x: self._check_pattern(x, data[’close’][stock]), raw=False

39 )

40 signal[stock] = combined_signal.fillna(0) # Fill NaNs with 0

41 # Note: Assumes the pattern’s absence on days where window_length is not
met

42

43 return signal

Rejected by mentor agent with the following suggestions:

1\. Refine the signal calculation to better distinguish between different
strengths of the pattern. Currently, it only returns a binary output (1 for
pattern detected, 0 otherwise), which lacks nuance for quantitative models.

2\. Consider adjusting the rolling window logic to ensure that only the
necessary number of days (exactly 3) are evaluated for the pattern, to improve
computational efficiency

3\. Incorporate an additional condition or metric to evaluate the strength and
reliability of the pattern, such as volume or the size of the candlesticks, to
provide a more informative and differentiated signal.

Trading Signal implementation 2nd version:

⬇

1class ImprovedThreeSoldierSignal(Factor):

2 def __init__(self):

3 self.name = "ImprovedThreeSoldierSignal"

4 # Adjusting window length to exactly 3 days as required for the pattern

5 self.window_length = 3 # Only the 3 days relevant for the pattern

6 self.inputs = ["close", "open", "volume"] # Including volume as a condition
for improved signal strength

7

8 def _check_pattern(self, opens, closes, volumes):

9 """

10 Check the last three days for the Three Advancing White Soldiers pattern

11 and quantify its strength.

12 """

13 # Checking basic pattern criteria

14 higher_close_than_open = (closes.shift(-2) > opens.shift(-2)) &
(closes.shift(-1) > opens.shift(-1)) & (closes > opens)

15 consecutive_higher_closes = (closes.shift(-2) < closes.shift(-1)) &
(closes.shift(-1) < closes)

16

17 # Additional strength evaluations

18 increasing_volume = (volumes.shift(-2) < volumes.shift(-1)) &
(volumes.shift(-1) < volumes)

19

20 # Calculate signal strength based on normalized candlestick body size and
volume increase

21 if higher_close_than_open and consecutive_higher_closes:

22 pattern_strength = (closes - opens).mean() / (closes.max() - opens.min())

23 volume_strength = increasing_volume.mean()

24 overall_strength = (pattern_strength + volume_strength) / 2

25 return overall_strength

26 else:

27 return 0

28

29 def calc(self, data):

30 # Initialize signal DataFrame

31 signal = pd.DataFrame(index=data[’close’].index,
columns=data[’close’].columns)

32

33 # Iterate over columns (stocks) and calculate the improved signal

34 for stock in data[’close’].columns:

35 combined_signal =
data[’open’][stock].rolling(window=self.window_length).apply(

36 lambda x: self._check_pattern(x, data[’close’][stock],
data[’volume’][stock]), raw=False

37 )

38 signal[stock] = combined_signal.fillna(0) # Fill NaNs with 0

39 # Note: Assumes the pattern’s absence on days where window_length is not
met

40

41 return signal

Rejected by mentor agent with the following suggestions:

1\. The ’-check-pattern’ function returns 0 when the pattern does not match,
which causes a lack of differentiation between the absence of the pattern and
weak patterns. Consider returning a negative value for non-patterns to
differentiate from weak patterns. 2\. In the ’-check-pattern’ function,
’higher-close-than-open’ and ’consecutive-higher-closes’ are boolean
conditions. Their calculation does not account for vectorized operations
correctly because the ’if’ condition expects a singular truth value. Use
aggregate functions (.all()) to handle series of booleans.” 3\. The
calculation of ’pattern-strength’ and ’volume-strength’ might result in
unbound values, especially ’volume-strength’ as ’increasing-volume.mean()’ can
be misleading because ’increasing-volume’ is a boolean condition. Refine
strength calculations to ensure meaningful signal values

Trading Signal implementation 3rd version:

⬇

1class ThreeSoldierSignalV3(Factor):

2 def __init__(self):

3 self.name = "ThreeSoldierSignalV3"

4 self.window_length = 3

5 self.inputs =["close", "open", "volume"]

6

7 def _calculate_pattern_strength(self, opens, closes, volumes):

8 """

9 Calculates the strength of the pattern based on closing prices,opening
prices,and volume

10 """

11 # Criteria evaluation

12 higher_close_than_open = (closes > opens).all()

13 consecutive_higher_closes =(closes.diff().iloc[1:] > 0).all()

14 volume_increasing = (volumes.diff().iloc[1:] > 0).all()

15

16 # Signal strength calculation criteria

17 if higher_close_than_open and consecutive_higher_closes:

18 close_open_ratio = ((closes-opens) / opens.sum()).sum()

19 volume_ratio = np.log(volumes.iloc[-1] / volumes.iloc[0]) if
volume_increasing else 8

20 # Adjusted overall strength based on magnitude of movements and volume
changes

21 overall_strength=(close_open_ratio * 0.7) + (volume_ratio * 0.3)

22 # Ensuring the strength lies between -1 and 1

23 overall_strength = max(min(overall_strength, 1), -1)

24 return overall_strength

25 else:

26 # Differentiating from weak patterns by returning a slightly negative value

27 return -0.1 # Indicates absence of pattern

28 def calc(self, data):

29 # Initialize signal DataFrame

30 signal = pd.DataFrame(index=data[’close’].index,
columns=data[’close’].columns)

31 # Iterate over columns(stocks)and calculate the finalsignal

32 for stock in data[’close’].columns:

33 combined_signal =
data[’open’[stock].rolling(window=self.window_length).apply(

34 lambda x: self._calculate_pattern_strength(x, data[’close’][stock],
data[’volume’][stock]),

35 raw=false

36 )

37 signal[stock]= combined_signal.fillna(-0.1)# Filling NaNs with -0.1 to
differentiate from absence of pattern

38

39 return signal

40 \end{minted}

mentor agent passed this implementation with the final review: The
implementation of the ThreeSoldierSignalV3 trading signal is quite sound,
capturing the essence of the candlestick pattern of the trading idea. Its
logic analyzes a three-day window for each stock, focusing on whether the
close prices are higher than the open prices, if the closing prices are
consecutively higher, and if trading volume is increasing, which are crucial
factors for identifying this bullish pattern. The unique approach of
calculating the pattern’s strength based on the magnitude of price changes and
volume variations, then balancing these factors to determine an overall
strength score between -1 and 1, effectively captures the signal’s intent. The
use of logarithmic scaling for volume change and the thoughtful consideration
of filling NaN values with -0.1 to signify the absence of the pattern
demonstrates a careful and innovative handling of the signal’s nuances