#  BLAZE: Cross-Language and Cross-Project Bug Localization via Dynamic
Chunking and Hard Example Learning

Partha Chakraborty, , Mahmoud Alfadel, , and Meiyappan Nagappan All authors
are from David R. Cheriton School of Computer Science, University of Waterloo,
Waterloo, Canada, N2L 3G1.
E-mail:{p9chakra, malfadel<EMAIL_ADDRESS>0000-0001-5965-615X
0000-0002-2621-6104 0000-0003-4533-4728

###### Abstract

Software bugs require developers to exert significant effort to identify and
resolve them, often consuming about one-third of their time. Bug localization,
the process of pinpointing the exact source code files that need modification,
is crucial in reducing this effort. Existing bug localization tools, typically
reliant on deep learning techniques, face limitations in cross-project
applicability and effectiveness in multi-language environments.

Recent advancements with Large Language Models (LLMs) offer detailed
representations for bug localization. However, they encounter challenges with
limited context windows and mapping accuracy. To address these issues, we
propose BLAZE, an approach that employs dynamic chunking and hard example
learning. First, BLAZE dynamically segments source code to minimize continuity
loss. Then, BLAZE fine-tunes a GPT-based model using challenging bug cases, in
order to enhance cross-project and cross-language bug localization. To support
the capability of BLAZE, we create the BeetleBox dataset, which comprises
26,321 bugs from 29 large and thriving open-source projects across five
different programming languages (Java, C$++$, Python, Go, and JavaScript). Our
evaluations of BLAZE on three benchmark datasets—BeetleBox, SWE-Bench, and Ye
et al.—demonstrate substantial improvements compared to six state-of-the-art
baselines. Specifically, BLAZE achieves up to an increase of 120% in Top 1
accuracy, 144% in Mean Average Precision (MAP), and 100% in Mean Reciprocal
Rank (MRR). An extensive ablation study confirms the contributions of our
pipeline components to the overall performance enhancement.

###### Index Terms:

Bug Localization, Embedding, Contrastive Learning

##  I Introduction

Software bugs are deviations from intended functionality in software
applications, leading to undesired outcomes [1]. Resolving these bugs requires
substantial developer effort, mainly to identify the exact source code file
that needs correction. Prior research indicates that debugging consumes about
one-third of developer work time [2, 3]. The challenge is exacerbated by the
often overwhelming number of bug reports compared to the available developer
resources, which can negatively affect customer satisfaction [4, 5].

Bug localization involves pinpointing the specific source code files that need
modification based on the details provided in bug reports [6]. Effective bug
localization tools minimize the time required to identify the root causes of
defects by automating the tracing of errors to their sources [7].

Previous works (e.g., [8, 9, 10, 11, 12]) have introduced bug localization
tools that measure the similarity between source code files and bug reports.
These tools, which often employ deep learning techniques like Word2Vec or
GloVe embeddings, are effective but heavily reliant on labeled data and
generally require retraining for each new project, limiting their cross-
project applicability [13, 14]. Although advancements have been made with
tools that use transfer learning to function across different projects (e.g.,
[15, 16]), they still require labeled data from new projects for retraining.
Moreover, these tools have predominantly been tested within single-language
environments. This raises concerns about their effectiveness in cross-project
bug localization involving multiple programming languages.

Recent studies have highlighted that Large Language Models (LLMs) provide
detailed and context-rich representations for code retrieval tasks and bug
diagnosis [17, 18]. This improved representation can enhance the ability of
bug localization tools to accurately link bug reports with the corresponding
source code files by integrating context-sensitive semantic understanding [19,
20]. However, such methods still encounter two major challenges. Firstly
(Challenge 1: Limited Context Window), the typical context window of LLMs,
which ranges from 512 to 2,048 tokens, is insufficient when compared to the
average token length of 2,536 for a single file in common datasets [21].

Secondly (Challenge 2: Mapping Accuracy), such methods struggle to accurately
map bug reports to the relevant buggy source code file, as illustrated in
Figure 1(a), where the ideal non-overlapping distribution between buggy and
non-buggy source code is not achieved. For this challenge, we highlight the
need to prioritize the importance of source code files within projects.
Consider the scenario depicted in Figure 2, where a font issue involves four
files. The model erroneously associates the bug with ‘ThemeManager.java,’ an
irrelevant file, and misses ‘FontController.java,’ which is crucial to the
issue. Focused learning from such challenging examples can improve the model,
aligning the representations of buggy source code files more closely with the
bug report.

To overcome the above two challenges, we propose BLAZE, an approach that
employs:

  * -

Dynamic Chunking: Using dynamic programming, BLAZE segments source code at the
boundaries of source code component declarations (e.g., class, interface, and
method) to minimize continuity loss and adhere to context length constraints.
We design these segments to be placed at component edges to prevent
overlapping, resolving issues of continuity loss and excessive resource
consumption caused by repetitive tokens.

  * -

Hard Example Learning: BLAZE employs a GPT-based bug localization model
tailored for identifying bugs across multiple languages and projects. We fine-
tune it by prioritizing complex bugs (hard examples). BLAZE identifies such
cases while tuning out less relevant outliers, which enhances the ability of
BLAZE to generalize across different projects and languages and identifies
critical misclassifications [22, 23]. As demonstrated in Figure 1(b), we find
that the fine-tuning method could minimize the similarity overlap between
buggy and non-buggy code files.

(a) Overlapped distribution before fine-tuning. (b) Minimally overlapped
distribution after fine-tuning (BLAZE). Figure 1: Kernel Density Estimate plot
showing the distribution of similarities between buggy and non-buggy source
code files and bug reports.

To evaluate the capability of BLAZE across languages, we compile BeetleBox,
the most extensive cross-language and cross-project bug localization dataset
to date, to the best of our knowledge. BeetleBox comprises 26,321 bugs that we
mine from 29 real-world and thriving projects across five widely used
programming languages: C++, Java, JavaScript, Python, and Go.

We evaluate the performance of BLAZE using three bug localization benchmarks,
namely SWE-Bench [24], Ye et al. [21], and BeetleBox, which together encompass
31,745 bugs from 36 real-world software projects. BLAZE achieves up to 141%
better Top 1 accuracy and 107.69% better MAP than three traditional cross-
project bug localization tools [15, 5, 16]. Compared to modern language model-
based tools [19, 20, 25], BLAZE achieves improvements of up to 63% in Top 1
and 69% in MAP. Moreover, we conduct an extensive ablation study, which
confirms that our newly designed pipeline and fine-tuning techniques enhance
performance. The results show that different components of our pipeline
substantially contribute to improving bug localization performance to
different degrees.

Contributions. In summary, this paper makes the following key contributions:

  * •

We introduce a novel bug localization technique (i.e., BLAZE). In this
technique,

    * –

We propose a dynamic chunking technique to overcome the limitation of
transformer-based approaches (limited context window).

    * –

We fine-tune BLAZE by learning from hard examples. BLAZE focuses on complex
and frequently misclassified cases to enhance model performance in cross-
language, cross-project settings.

  * •

We create BeetleBox, the largest to-date cross-language and cross-project bug
localization dataset.

  * •

We conduct an ablation study to understand the importance of each component of
the BLAZE pipeline.

Fostering open science. To foster future advances in bug localization, we
release our dataset and we make the scripts used for our experiments publicly
available [26].

Figure 2: Example of mining hard examples.

##  II Methodology

In this section, we describe our approach to designing BLAZE, which consists
of three phases: Phase A introduces a fine-tuned language model to associate
textual (bug report) and code elements (source code files). Phase B implements
a dual-indexing system for source code files using both text and vector data
from the fine-tuned model to facilitate a more effective retrieval of relevant
files. Finally, in Phase C, we leverage the fine-tuned model from Phase A and
the resulting indexed databases from Phase B together to facilitate the full
process of matching incoming bug reports with the corresponding source code
files. Below, we describe each phase in detail.

###  Phase A Fine-tuning of Language Model

Figure 3: Fine-tuning of language model (Phase A).

Figure 3 presents the steps we follow to fine-tune the model used in BLAZE.
Each step of this process is detailed below.

Extraction of Source Code Elements. Our process begins with obtaining bug
reports, the specific repository version where the bug exists, and all the
source code files from that version provided in the targeted dataset (more
details on how we collect and process the dataset in this study are described
in Section III.B). In this initial step, we focus on extracting details such
as the names and roles of components (e.g., methods, classes, and interfaces)
from the collected source code files. Drawing on insights from prior research
[27], which indicates that separate indexing of source code files enhances
lexical retriever’s performance in code search tasks, we index source code
files separately in the next phase of our pipeline (Phase B).

Moreover, we extract positional information, such as line numbers, which is
essential for subsequent analysis stages. To identify all components within
the source code, we use tree-sitter,111https://github.com/tree-sitter/tree-
sitter a tool that structures code into a graph format. From this tree-sitter
graph, we extract all components along with their types, positions (line
numbers), and names.

Input :

cost_map: Map containing costs for each code component type.

total_lines: Total number of lines.

window_size: Maximum chunk size.

Output :

dp: Array containing minimum costs and corresponding breakpoints

1 Function  _dynamic_chunk(cost_map, total_lines, window_size)_

2 line_map $\leftarrow$ get line number to component type map

3 split_cost_map $\leftarrow$ get line number to cost map

4 for  _$i\leftarrow$ 1 to total_lines_ do

5 minimum cost $\leftarrow$ $INFINITY$

6 breakpoint $\leftarrow$ $-1$

7 for  _$j\leftarrow$ MAX(i - window size, 0) to $i$_ do

8 cost = dp[$j$][0] + split_cost_map[$i$] or DEFAULT_COST

9 if  _cost $<$ minimum_cost_ then

10 minimum_cost = cost

11 breakpoint = $j$

12

13

14 dp[$i$] = (minimum_cost, breakpoint)

15 return dp

16

Algorithm 1 Dynamic Chunking.

Dynamic chunking of source code file. In this step, we address the challenge
of segmenting lengthy source code files into smaller segments that are suited
for context-aware embedding models [28, 29, 30]. We employ a dynamic
programming approach for chunking. This method segments files at natural
boundaries (e.g., classes, interfaces, and methods), ensuring continuity and
preventing overlaps—issues commonly associated with the traditional static
chunking method and the sliding window technique.

Our algorithm, outlined in Algorithm 1, details the pseudo-code for our
dynamic chunking approach. In the algorithm, we define ‘DEFAULT_COST’ as the
cost of a random line. The approach begins by assigning each line of code a
specific cost and categorizing it by component type. It then calculates the
least expensive way to segment the code up to each line iteratively. It
evaluates all potential breakpoints within the predefined window size (maximum
context window supported by the intended model), updates to the minimum cost
when a less expensive option is identified, and records the optimal
breakpoint. The final output of the algorithm includes a list of the minimal
costs associated with each breakpoint, alongside the breakpoints themselves.
These are used to divide the source code files into smaller chunks.

Learning from hard examples. We fine-tune a Large Language Model (LLM), called
CodeSage [31]. CodeSage is a GPT-based multi-modal embedding model. We adopt
CodeSage to align bug reports and source code files within the same embedding
space, drawing on its proven effectiveness in zero-shot code search tasks
[31]. To optimize CodeSage for bug localization, we implement contrastive
learning, which helps to distinguish between similar and dissimilar data
pairs, i.e., this technique adjusts the model to minimize the distance between
embeddings of bug reports and their corresponding buggy files (positive) while
maximizing the separation from non-buggy files (negative). This distinction is
guided by the Normalized Temperature-scaled Cross-Entropy (NT-Xent) loss,
defined as:

$\mathcal{L}_{i,j}=-\log\frac{\exp^{sim(z_{i},z_{j})/\tau}}{\sum_{k=1}^{N}1_{[k\neq
i]}\exp^{sim(z_{i},z_{k})/\tau}}$ (1)

where $z_{i}$ is the embedding of the bug report, $z_{j}$ is the embedding of
the positive (buggy) source code file, and $\tau$ is the temperature parameter
that scales the similarity. This loss function distinguishes between positive
and negative examples by comparing similarity scores.

However, the standard NT-Xent loss may fall short in distinguishing between
positive and negative source code files, potentially causing a bug report to
align closer with a negative file. To mitigate this, we incorporate in-batch
hard example mining into our training loop. This strategy prioritizes the most
challenging cases: positive files with low similarity to the bug report and
negative files with unexpectedly high similarity. We identify these hard
examples by calculating the median similarity of all pairs in each batch.
Negative source code files with similarity above this median and positive
source code files below the median are considered hard examples. These cases
are then given priority in the training process, with their impact on the loss
function adjusted as follows:

$\displaystyle\begin{gathered}\mathcal{L}_{i,j}=-\log\frac{M_{i,j}*\exp^{sim(z_{i},z_{j})/\tau}}{\sum_{k=1}^{N}1_{[k\neq
i]}\exp^{sim(z_{i},z_{k})/\tau}*M_{i,k}}\\\
M_{m,n}=\begin{cases}\alpha,&\text{\parbox[t]{130.08731pt}{if $z_{n}$ negative
and $sim(z_{m},z_{n})$ $>$ $median(sim(z_{u},z_{v}))$}}\\\
\beta,&\text{\parbox[t]{130.08731pt}{if $z_{n}$ positive and
$sim(z_{m},z_{n})$ $<$ $median(sim(z_{u},z_{v}))$}}\\\
1,&\text{otherwise}\end{cases}\\\ \alpha>1;\beta>1;1\leq u,v\leq
N\end{gathered}$ (5)

In an average project, approximately 700 source code files exist, with only
1-2 directly related to a reported bug, as evidenced in Apache project dataset
[21]. This rarity of relevant files presents a considerable challenge in bug
localization. Our approach of hard example mining and scaling enhances the
gradients from challenging cases and helps maintain a balanced learning
environment, preventing the model from being overwhelmed by outliers.
Moreover, the continuous nature of the scaling multiplier preserves the
convergence properties of the NT-Xent loss. By employing median-based scaling,
our method could improve the model learning from difficult examples, thereby
enhancing its convergence. Henceforth, we refer to the fine-tuned language
model as the BLAZE model.

Figure 4: Offline indexing phase of BLAZE (Phase B).

###  Phase B Offline Indexing

We introduce the offline indexing phase to enhance bug localization by
proactively processing and indexing source code files. As mentioned earlier,
repositories contain (on average) about 700 source code files, and without
this phase, developers could face delays due to real-time data processing.

As depicted in Figure 4, upon receiving a bug report, BLAZE employs two
distinct indexing strategies: the deep retriever stores data in the Milvus
vector database [32], while the lexical retriever uses the no-SQL database
ElasticSearch [33]. Having two types of retrievers can help to offset their
weaknesses and take advantage of their strengths. Note that this phase builds
upon the earlier phase of extracting names and positions of components (see
Phase A). Below, we describe each retriever in detail.

Indexing for lexical retriever. Lexical retrievers are fast and use keyword
matching to assess similarity. After extracting component names and positions
as detailed in Phase A), each source code file is indexed in ElasticSearch.
ElasticSearch leverages the Okapi BM25 algorithm [34] to compute similarity
scores, a technique well-documented in previous studies for applications in
code retrieval [27, 35]. Additionally, we index filenames and file paths to
match logs and stack traces within bug reports.

Indexing for deep retriever. Deep retrievers integrate semantics to calculate
the similarity between bug reports and source code files. Starting with the
dynamically chunked source code files—as outlined in Phase A—each chunk is
transformed into an embedding using the BLAZE model. These embeddings are then
indexed in Milvus, an open-source vector database specifically optimized for
storing and retrieving embedding vectors in artificial intelligence
applications.

###  Phase C Retrieval Pipeline

Figure 5: Retrieval phase of BLAZE (Phase C).

Figure 5 illustrates the retrieval phase of BLAZE. We leverage a combination
of lexical and deep retrievers along with the fine-tuned BLAZE model. As shown
in the figure, upon the arrival of a bug report, we query the Milvus and
ElasticSearch indices to pinpoint the relevant source code files. We then
integrate the rankings from both indices using a technique known as Reciprocal
Rank Fusion (RRF). Next, we provide a detailed description of these steps.

Retrieval using deep retriever. Deep retrievers have demonstrated superior
performance in ranking tasks compared to traditional vocabulary matching
approaches [36]. To use a deep retriever, we first calculate the embedding of
the bug report using our fine-tuned BLAZE model. Then, we retrieve the
embedding of all the source code files from Milvus, a vector index we built
during offline indexing. Milvus delivers the top K most similar chunks to the
bug report embedding. We use cosine similarity to assess the similarity
between embeddings. Finally, we sort the source code files based on decreasing
similarity.

Retrieval using lexical retriever. In the lexical retrieval process, we query
the ElasticSearch index with the bug report, which returns similarity scores
for each source code file based on lexical matches. We use these scores to
rank the source code files, leveraging ElasticSearch text searching
capabilities to localize files potentially containing the bug. By directly
using the bug report as a query, we ensure that the retrieval is tightly
focused on the textual content relevant to the bug.

Reciprocal rank fusion. While deep retrievers demonstrate superior performance
in capturing semantic similarity, they sometimes struggle with lengthy
documents and may face challenges such as domain shifts when applied across
different projects [37, 38]. These shifts can lead to outdated understandings
of new domains. Conversely, lexical retrievers are immune to domain shifts but
can encounter difficulties with vocabulary mismatches due to their lack of a
semantic understanding. To address these limitations and harness the strengths
of both retrieval types, we employ Reciprocal Rank Fusion (RRF) [39]. RRF
integrates the rankings from both the lexical and deep retrievers by
considering the position of each source code file in the rankings rather than
merging their numerical scores. This method, supported by prior research [39,
40, 41], has proven to be a reliable and powerful technique for creating
robust ensembles that enhance overall retrieval performance.

Given a bug report $r$, a source code file $s$, and a set of retrievers $M$,
where $\pi^{m}(r,s)$ represents the ranking by retriever $m$ for source code
file $s$ and bug report $r$, the RRF score is defined as follows:

$RRF(r,s,M)=\sum_{m\epsilon M}\frac{1}{k+\pi^{m}(r,s)}$ (6)

Here, $k$ is a hyperparameter. Based on prior studies [39, 41], $k=60$ has
been found to provide the highest mean average precision (MAP). We also
experimented with $k$ values ranging $\epsilon[50,70]$ in the BeetleBox
dataset and confirmed that $k=60$ continues to yield the highest MAP.

##  III Experimental Design

In this section, we present the Research Questions (RQs) we tackle in our
study, the process of our dataset collection, the subject baselines, and the
performance metrics employed to evaluate the performance of BLAZE.

###  III.A Research Questions

Below, we present our RQs by explaining the motivation behind each one.

RQ1. How effective is BLAZE compared to state-of-the-art cross-project bug
localization tools?
A plethora of bug localization tools have been proposed (e.g., [12, 42, 43,
16]), with cross-project tools (e.g., [15, 16]) being particularly effective.
However, these tools still require project-specific additional training. Our
proposed approach, BLAZE, is designed to function without this requirement.
This RQ evaluates how well BLAZE performs compared to state-of-the-art cross-
project bug localization tools.

RQ2. How effective is BLAZE compared to state-of-the-art embedding-based bug
localization tools?
Embedding-based techniques, which assess document similarity, are extensively
used for bug localization. These techniques differ from traditional bug
localizers that provide binary predictions by producing embeddings and
employing metrics like cosine distance to evaluate if a source code file
contains bugs. Their ability to contextualize associations between bug reports
and source code enhances their effectiveness. This RQ explores how well BLAZE
performs compared to these embedding-based tools.

###  III.B Dataset Curation of BeetleBox

We have compiled an extensive multi-language, multi-project bug localization
dataset. Previous research [21, 5, 44, 45] has proposed various datasets for
bug localization, but these have numerous shortcomings, such as support for
only one programming language and a limited range of projects. Additionally,
some datasets, such as Bench4BL [46], contain inaccuracies in their ground
truth data [47]. To address these challenges, we introduce BeetleBox. Below,
we detail the process we follow for curating BeetleBox.

  * •

Selection of repositories. We use the GitHub API search feature to collect
repository names. We search for repositories written in the top five
programming languages, Java, Python, C$++$, JavaScript, and Go, as ranked in
the StackOverflow Survey.222https://survey.stackoverflow.co/2022/#technology-
most-popular-technologies We select the top 10 repositories for each language,
sorting them in descending order by their star ratings and most recent
updates. This step guarantees that the repositories are actively contributed
to and popular within the open-source community.

  * •

Linking pull-requests and issues. We gather ‘closed’ issues for each
repository and their corresponding fixing Pull Request (PR). We identify these
PRs by using predefined GitHub
keywords.333https://docs.github.com/en/issues/tracking-your-work-with-
issues/linking-a-pull-request-to-an-issue

  * •

Filtration of issues. An issue may have several PRs, but typically only one is
merged. To minimize errors in our ground truth, we filter out issues with the
‘duplicate’ tag and verify using GitHub API that each issue is linked to a
single, merged PR. We initially started with 60 repositories. However, in
certain projects, we found no issues meeting all the criteria. In the end, we
are left with 29 repositories across five programming languages.

For each bug report in BeetleBox, the dataset, we include the bug status,
repository name, repository URL, issue ID, a list of files modified during the
fix, the title of the bug report, its body, URL link to the merged PR, URL
link to the issue, SHA values of before and after the fix, the date and time
the bug was reported, and the date and time of the fixing commit. Overall,
BeetleBox comprises 26,321 bugs extracted from 29 projects. Detailed
statistics about BeetleBox and a list of the projects can be found in the
online Appendix [26].

Finally, to assess the accuracy of the ground truth in BeetleBox, we use the
method outlined by Kim et al. [47]. Our manual analysis reveals that BeetleBox
has an error rate of 0.06%, with a 95% confidence level.

###  III.C Evaluation Benchmarks and Baselines

Benchmarks. In addition to our BeetleBox dataset, we employ SWE-Bench [24],
and Ye et al. [21]. Similar to BeetleBox, Ye et al. dataset is also recognized
for its low error rate in its ground truth. While BeetleBox and Ye et al. are
solely bug localization datasets, SWE-Bench is a recent dataset that is a
benchmarking suite for bug localization as well as automatic patch generation.

TABLE I: Statistics on the used datasets, showing the distribution of bugs
across training and testing, categorized by language for each dataset.

Dataset | Purpose | Language | # Bugs
---|---|---|---
BeetleBox | train | c$++$ | 3,868
go | 758
java | 3,369
javascript | 1,974
python | 3,215
test | c$++$ | 4,783
go | 400
java | 2,270
javascript | 3,085
python | 2,599
SWE-Bench [24] | test | python | 2,294
Ye et al. [21] | test | java | 16,314

In our methodology, we fine-tune the language model using the training split
of BeetleBox. For evaluation, we use the test splits of BeetleBox, SWE-Bench,
and the dataset from Ye et al. Table I provides brief statistics of these
three datasets.

Studied baselines. We compare BLAZE with state-of-the-art cross-project bug
localization tools, TRANP-CNN [15] and CooBa [16], which both use few-shot
learning for cross-project adaptation. TRANP-CNN, a CNN-based deep learning
tool, and CooBa, which uses dual encoders, train on a source project and are
fine-tuned on a target project with limited samples. We also include
BugLocator [5] as a baseline in our evaluation. Although not designed for
cross-project use, BugLocator, like BLAZE, is an information retrieval-based
tool that does not require additional training for deployment in new projects.
The characteristics make it a relevant comparison in our evaluation.

In addition, we evaluate the performance of BLAZE compared to embedding-based
bug localizers, namely SemanticCodeBERT [20] and FBL-BERT [19]. Both are BERT-
based models, where SemanticCodeBERT incorporates semantic flow graph
information into its training, and FBL-BERT incorporates changeset data.
Recent advancements in generative AI have also introduced models like _openai-
embedding_ [25], which have demonstrated superior performance in tasks
involving natural language and code. Therefore, OpenAI embedding is also
included in our baseline for comparison.

###  III.D Evaluation Metrics

We adopt the following metrics that are widely adopted for evaluating bug
localization tools [15, 43, 5]: Top N, Mean Reciprocal Rank (MRR), and Mean
Average Precision (MAP).

  * •

Top N. Top N measures the overall ranking performance of the bug localization
model. It indicates the percentage of bug reports for which at least one buggy
source code file appears among the top N positions in the ranked list
generated by the bug localization tool. Following previous studies [43, 15],
we consider three values of N: 1, 5, and 10.

  * •

Mean Reciprocal Rank (MRR). MRR measures the average rank of the relevant file
in the retrieved files set. It calculates the average reciprocal rank of the
relevant source code files across all bug reports. The following equation
calculates MRR, where $A$ represents the set of bug reports.

$MRR=\frac{1}{|A|}\sum_{A}\frac{1}{Least\ rank\ of\ the\ relevant\ files}$ (7)

For each bug report, our bug localization model ranks the source code files.
Let us consider two bug reports, $report_{1}$ and $report_{2}$, each with six
candidate source code files ranked by the model. For $report_{1}$, the ground
truth of the retrieved files are $[0,0,1,0,1,0]$, and for $report_{2}$, the
ground truth of the retrieved files are $[1,0,0,0,0,1]$. In this case, the
least rank of relevant files is 3 and 1, respectively, for $report_{1}$ and
$report_{2}$. Therefore, $MRR=\frac{1}{2}(\frac{1}{3}+\frac{1}{1})=0.67$.

  * •

Mean Average Precision (MAP). MAP measures the retrieval quality in scenarios
where a bug is linked to multiple source code files. MAP considers the rank of
all relevant files in the retrieved files list, making it a more comprehensive
and unbiased metric than MRR. To compute MAP, we determine precision@1,
precision@2, …, and precision@k; then, we calculate the average precision at
different points. After calculating the average precision for each bug report,
we calculate the mean of the average precision to compute MAP. The equation
for calculating MAP is presented below, where $A$ represents the set of bug
reports.

$MAP=\frac{1}{|A|}\sum_{A}{AvgPrecision(Report_{i})}$ (8)

To demonstrate the computation of MAP, consider the same example of two bug
reports as previously discussed. The average precision calculated for
$report_{1}$ is 0.37, while it is 0.67 for $report_{2}$. Thus,
$MAP=\frac{1}{2}(0.36+0.67)=0.52$.

###  III.E Configuration Setup

We fine-tune the BLAZE model in a distributed setting with eight cores, 128GB
of RAM, and fourteen Nvidia V100 32GB GPUs. We conduct inference on the same
server using an Nvidia V100 32 GB GPU, which allows for fast and efficient
predictions. We use PyTorch v.2.1.2, PyTorch-lightning v2.2.0, and the
HuggingFace library v.4.36.2 to fine-tune our model. We also use ElasticSearch
v8.7.1 and Milvus v2.3.4 for efficient similarity search. We fine-tune our
language model on the BeetleBox dataset for 15 epochs, with batches of size 80
and a decaying learning rate starting at 2E-07, as recommended in prior work
[28, 19]. More details about the hyperparameters are available in our online
Appendix [26].

##  IV Results

In this section, we present our results per each RQ.

### RQ1: Comparison to Cross-Project Tools.

TABLE II: Comparative performance of BLAZE and cross-project bug localization
tools.

Dataset | Model Name | Top 1 | Top 5 | Top 10 | MAP | MRR
---|---|---|---|---|---|---
BeetleBox | BLAZE | 0.18 | 0.37 | 0.45 | 0.22 | 0.27
TRANP-CNN | 0.08 | 0.13 | 0.19 | 0.09 | 0.13
CooBa | 0.06 | 0.1 | 0.14 | 0.08 | 0.09
BugLocator | 0.06 | 0.10 | 0.15 | 0.08 | 0.12
SWE-Bench | BLAZE | 0.29 | 0.68 | 0.79 | 0.43 | 0.45
TRANP-CNN | 0.13 | 0.28 | 0.42 | 0.26 | 0.34
CooBa | 0.12 | 0.25 | 0.38 | 0.21 | 0.3
BugLocator | 0.1 | 0.14 | 0.21 | 0.17 | 0.21
Ye et al. | BLAZE | 0.16 | 0.35 | 0.44 | 0.21 | 0.26
TRANP-CNN | 0.09 | 0.17 | 0.23 | 0.14 | 0.15
CooBa | 0.08 | 0.13 | 0.19 | 0.09 | 0.13
BugLocator | 0.09 | 0.11 | 0.16 | 0.07 | 0.12

Approach. We train our model using BeetleBox. We conduct the training on 13K
bugs from five languages, covering 13 projects. Our evaluation dataset
contains 31,745 bugs from five languages covering 29 real-world software
projects. Following prior studies [24, 48], BeetleBox uses a completely
different set of projects for training and testing splits. As previously
presented in Table I, the SWE-Bench and Ye et al. datasets are used only for
testing.

Result. Table II presents the performance of BLAZE compared to TRANP-CNN,
CooBa, and BugLocator, across the studied datasets. Overall, BLAZE
demonstrates its highest performance in the SWE-Bench dataset. Conversely, its
performance is reduced in the Ye et al. dataset, as indicated by both Top 1
and MAP scores. Nonetheless, BLAZE consistently outperforms comparative
baselines in this dataset and across all evaluated datasets.

In the BeetleBox dataset, BLAZE surpasses TRANP-CNN, with improvements ranging
between 107% to 144.44%. Specifically, BLAZE achieves a 125% increase in Top 1
accuracy, a 144.44% increase in MAP, and a 107.69% in MRR. When compared to
CooBa, BLAZE shows substantial gains: 200% in Top 1, 175% in MAP, and 200% in
MRR. Against BugLocator, BLAZE shows improvements of 200% in Top 1, 175% in
MAP, and 125% in MRR.

In the SWE-Bench dataset, BLAZE performs better than TRANP-CNN with increases
between 32% and 123% across various metrics. The enhancements include a 123%
improvement in Top 1, 65% in MAP, and 32% in MRR. Against CooBa, BLAZE shows a
141% increase in Top 1, 104% in MAP, and 50% in MRR. Compared to BugLocator,
BLAZE shows improvements of 190% in Top 1, 52.9% in MAP, and 61.9% in MRR.

In the Ye et al. dataset, BLAZE shows an improvement over TRANP-CNN with gains
of 77% in Top 1, 50% in MAP, and 73% in MRR. In comparison to CooBa, BLAZE
shows increases of 100% in Top 1, 133% in MAP, and 100% in MRR. Against
BugLocator, the model posts gains of 77.7% in Top 1, 200% in MAP, and 25% in
MRR.

Additionally, across the studied programming languages, BLAZE achieves Top 1
scores of 0.18 in C$++$, 0.36 in Go, 0.26 in Java, 0.21 in JavaScript, and 0.3
in Python. These results show an improvement of 50.9% to 135.6% compared to
TRANP-CNN, with the highest improvement observed in JavaScript and the lowest
in Python. The trend is similar to that of CooBa, with improvements ranging
from 82% to 162%, with the highest increase observed in C$++$ and the lowest
in Python. Against BugLocator, the improvement ranges from 87% to 160%, with
the highest improvement in JavaScript and the lowest in Python.

### RQ2: Comparison to Embedding-Based Tools.

TABLE III: Comparative performance of BLAZE and embedding-based bug
localization tools.

Dataset | Model Name | Top 1 | Top 5 | Top 10 | MAP | MRR
---|---|---|---|---|---|---
BeetleBox | BLAZE | 0.18 | 0.37 | 0.45 | 0.22 | 0.27
OpenAI | 0.13 | 0.26 | 0.32 | 0.1 | 0.19
SemanticCodeBERT | 0.11 | 0.20 | 0.31 | 0.13 | 0.21
FBL-BERT | 0.12 | 0.23 | 0.30 | 0.16 | 0.19
SWE-Bench | BLAZE | 0.29 | 0.68 | 0.79 | 0.43 | 0.45
OpenAI | 0.27 | 0.63 | 0.72 | 0.36 | 0.39
SemanticCodeBERT | 0.23 | 0.5 | 0.64 | 0.31 | 0.33
FBL-BERT | 0.20 | 0.57 | 0.51 | 0.31 | 0.30
Ye et al. | BLAZE | 0.16 | 0.35 | 0.44 | 0.21 | 0.26
OpenAI | 0.17 | 0.36 | 0.43 | 0.19 | 0.25
SemanticCodeBERT | 0.13 | 0.24 | 0.30 | 0.17 | 0.22
FBL-BERT | 0.10 | 0.24 | 0.28 | 0.14 | 0.18

Approach. We examine three embedding-based bug localization tools:
SemanticCodeBERT [20], FBL-BERT [19], and OpenAI [25]. For
SemanticCodeBERT444https://github.com/IBM/semanticflowgraph and FBL-
BERT555https://anonymous.4open.science/r/fbl-bert-700C/README.md, we use their
publicly available replication package to train the models and produce
embeddings. For OpenAI, we obtain embeddings for both bug reports and source
code files using the OpenAI API.666https://openai.com/index/openai-api/ We
employ cosine similarity to rank the source code files based on their
similarity scores, facilitating the identification of the buggy files.

Result. Table III illustrates the performance of BLAZE relative to the studied
embedding-based tools. From the table, we observe trends similar to those
observed in Table II. We also observe that the performance across all models
is strongest in the SWE-Bench dataset, which comprises Python projects, and
weakest in the Ye et al. dataset, featuring Java projects.

In the BeetleBox dataset., BLAZE demonstrates superior performance to OpenAI,
with improvements ranging from 38% in Top 1 accuracy to 120% in MAP and 42.11%
in MRR. Similarly, when compared to SemanticCodeBERT, BLAZE achieves
substantial gains, with increases of 63% in Top 1, 69% in MAP, and 28% in MRR.
It also shows notable enhancements against FBL-BERT, including a 50% rise in
Top 1 and improvements in MAP and MRR by 37% and 42%, respectively.

The performance trend continues in the SWE-Bench dataset, where BLAZE
outperfomrs OpenAI with gains from 7% in Top 1 to 19% in MAP and 15% in MRR.
It also outperforms SemanticCodeBERT with a 26% increase in Top 1 and even
greater enhancements in MAP and MRR. Against FBL-BERT, BLAZE shows substantial
improvements of 45% in Top 1, 38% in MAP, and 50% in MRR.

In the Ye et al. dataset, BLAZE slightly lags behind OpenAI in Top 1 by a
margin of 5%, but it still achieves better performance in MAP and MRR by 10%
and 4%, respectively. It demonstrates enhancements over SemanticCodeBERT,
improving by 23% in both Top 1 and MAP, and 18% in MRR. Similarly, against
FBL-BERT, BLAZE achieves substantial performance increases of 60% in Top 1,
50% in MAP, and 44% in MRR.

When analyzing the performance of BLAZE across the studied programming
languages, we find that BLAZE consistently outperforms the studied models in
Top-1 scores across most programming languages. The exception is Go, where
OpenAI slightly outperforms BLAZE in Go projects. We find that Go projects
contain only 758 bugs, the fewest in our dataset. Additionally, the unique
concurrency model and error-handling patterns of Go may pose challenges for
BLAZE and contribute to lower performance in this language. Moreover, BLAZE
surpasses OpenAI with improvements ranging from 14.24% to 16.86% in all
languages except Go, where it lags by 0.66%. Compared to OpenAI, BLAZE
achieves the highest improvement in Python and the lowest improvement in
C$++$. Compared to SemanticCodeBERT, BLAZE shows consistent improvement
between 15% and 41%, with the highest being observed in JavaScript and the
lowest in Go. The trend continues when compared with FBL-BERT, where
improvements range from 34.1% to 67.5%, peaking in Java and being lowest in
JavaScript.

##  V Ablation Study

This section presents an ablation study to assess the influence of individual
components in the BLAZE pipeline on its overall performance. Key strategies
contributing to the effectiveness of BLAZE include (1) a combined pipeline of
lexical and deep retrievers, (2) learning from hard examples, and (3) dynamic
chunking. Next, we detail the results of the ablation study to evaluate the
impact of each component.

###  V.A Impact of hybrid retrieval technique

Figure 6: Precision recall curves of the retrievers. TABLE IV: Performance
comparison between Lexical, Deep, and Hybrid Retriever.

Retriver Type | Top 1 | Top 5 | Top 10 | MAP | MRR
---|---|---|---|---|---
Lexical | 0.07 | 0.12 | 0.19 | 0.11 | 0.13
Deep | 0.12 | 0.25 | 0.31 | 0.14 | 0.18
Hybrid (BLAZE) | 0.18 | 0.37 | 0.45 | 0.22 | 0.27

In BLAZE, we use a hybrid technique by combining outputs from two types of
retrievers: lexical and deep. The lexical retriever is typically useful at
finding exact matches and managing simple searches, while the deep retriever
is better suited for understanding complex queries through their semantics. We
assess their combined performance with a precision-recall graph, as shown in
Figure 6. The figure demonstrates how accurately BLAZE identifies buggy source
code files at various decision thresholds, with higher precision and recall
indicating better performance.

Figure 6 shows that the lexical retriever initially has high precision but low
recall, indicating it identifies relevant files well but may miss others. As
it retrieves more files, precision drops. Conversely, the deep retriever’s
precision increases as it processes more files, particularly at higher recall
levels. By combining both retrievers in the BLAZE pipeline (as denoted
‘Hybrid’ in Figure 6), we achieve a balance that enhances overall results.

Furthermore, we analyze the performance of each retriever and their combined
effect in the BLAZE pipeline on the BeetleBox dataset. From Table IV, we
observe that the deep retriever outperforms the lexical retriever, showing
improvements of 71% in Top 1, 27% in MAP, and 38% in MRR. Combining the
retrievers further enhances performance, showing improvements of 45% to 57%
over the deep retriever alone. This result demonstrates that combining both
retriever types mitigates their individual weaknesses and enhances the overall
performance of BLAZE.

###  V.B Impact of fine-tuning

(a) Epoch 0 (CodeSage)

(b) Epoch 5

(c) Epoch 15 (BLAZE)

Figure 7: Scatter plots showing the chunks of the buggy and non-buggy files
associated with the Apache Dubbo-1216 bug report.  • denotes bug report,  •
denotes chunks from buggy files, and  • denotes chunks from non-buggy files.

In BLAZE, we fine-tune a GPT-based model [31] to better target hard-to-
localize bugs by adjusting how closely buggy (positive files) and non-buggy
(negative files) source code relate to bug reports (see Phase A). We assess
the impact of this fine-tuning by comparing the performance of the model at
different stages: epoch 0 (CodeSage), epoch 5, and epoch 15 (BLAZE). We use
t-SNE [49], a method that simplifies complex data into two or three dimensions
for easier visualization, to generate and display embeddings for a specific
bug report (i.e., _Apache Dubbo-1216_) and it is corresponding source code.

Figure 7 shows that the ability of the CodeSage model to distinguish between
buggy and non-buggy files in the analyzed bug report (i.e., _Apache
Dubbo-1216)_ evolves over time. At the start (epoch 0), the model struggles to
differentiate, with overlapping positive and negative chunks. By epoch 5,
there is a clear improvement: positive chunks start clustering more closely,
and negative chunks spread out, although some overlap remains. By epoch 15,
the model has made substantial progress; positive chunks are well-grouped, and
no negative chunks are closer than the positives, indicating successful fine-
tuning.

TABLE V: Performance of CodeSage vs. BLAZE.

Retriver Type | Top 1 | Top 5 | Top 10 | MAP | MRR
---|---|---|---|---|---
CodeSageDeep | 0.11 | 0.21 | 0.28 | 0.12 | 0.16
CodeSageHybrid | 0.17 | 0.32 | 0.42 | 0.19 | 0.24
BLAZE Deep | 0.12 | 0.25 | 0.31 | 0.14 | 0.18
BLAZE Hybrid | 0.18 | 0.37 | 0.45 | 0.22 | 0.27

Also, we evaluate the CodeSage model on the BeetleBox dataset. Table V shows
the results of the CodeSage and BLAZE model with different types of
retrievers. For instance, CodeSageDeep reflects the performance with a deep
retriever, while CodeSageHybrid indicates the results using a hybrid retriever
combining deep and lexical retriever. Using only the deep retriever, BLAZE
achieves a performance improvement of 6.9%–12.5% over CodeSage, including 9.1%
for Top1, 16.7% for MAP, and 12.5% for MRR. The hybrid technique used by the
BLAZE pipeline leads to even more enhancements, ranging from 5.8% to 15.7%,
with gains of 5.8% in Top1, 15.8% in MAP, and 12.5% in MRR. These results
support that fine-tuning the model on challenging (hard) examples improves its
effectiveness.

###  V.C Impact of dynamic chunking

In BLAZE, we use dynamic chunking to manage the issue of limited context size
while reducing continuity loss. Previous research [50, 51] has used methods
like sliding windows and static chunking. The sliding window method overlaps
chunks to maintain continuity but complicates similarity calculations due to
these overlaps. Static chunking, on the other hand, avoids overlaps but can
lead to a loss of continuity. Our approach uses dynamic programming to find
breakpoints that reduce continuity loss without shrinking the window size. We
trained and evaluated BLAZE using all three chunking methods to see the
effects of each.

TABLE VI: Static vs. Sliding window vs. Dynamic chunking.

Model | Top 1 | Top 5 | Top 10 | MAP | MRR
---|---|---|---|---|---
BLAZE Static + Deep | 0.11 | 0.22 | 0.26 | 0.13 | 0.17
BLAZE Sliding + Deep | 0.11 | 0.24 | 0.3 | 0.13 | 0.17
BLAZE Dynamic + Deep | 0.12 | 0.25 | 0.31 | 0.14 | 0.18
BLAZE Static + Hybrid | 0.15 | 0.33 | 0.37 | 0.18 | 0.25
BLAZE Sliding + Hybrid | 0.17 | 0.36 | 0.42 | 0.21 | 0.25
BLAZE Dynamic + Hybrid | 0.18 | 0.37 | 0.45 | 0.22 | 0.27

Table VI presents a performance comparison of various chunking techniques. For
instance, BLAZEStatic + Deep represents the performance of static chunking
paired with the deep retriever. The table shows that BLAZEDynamic + Deep
improves performance by 5.5–16% over static chunking and by 3.23–8.33% over
the sliding window method. Within the hybrid technique of BLAZE pipeline,
dynamic chunking shows a performance increase of 7–18% compared to static
chunking and 2–7.41% compared to sliding window.

##  VI Threats to Validity

Below, we discuss the threats to the validity of our study.

Internal validity. One threat to internal validity is the hyperparameters we
used to train the models. To mitigate this, we use the same parameters used by
several prior studies that use large language models [28, 19, 52, 13].

External validity. A threat to the external validity of our study lies in the
generalizability of the results. We assess cross-project capabilities using
separate projects for training and testing. Although the choice of test
projects may affect performance, this approach aligns with previous studies
that evaluate zero-shot and cross-project capabilities [48, 24]. Additionally,
our evaluation of 31,745 real-world bugs ensures that our findings hold, even
with different project selections, due to the large scale of our dataset. To
address potential selection bias in the BeetleBox dataset, we collected
project repositories based on star ratings and recent updates, ensuring that
the repositories are actively contributed to and popular within the open-
source community. This approach aims to include high-quality, widely used
projects across the selected top five programming languages.

Construct validity. Our selected evaluation metrics can threaten the construct
validity of the study. We used Top N, MAP, and MRR as evaluation metrics.
These are well-known evaluation measures adopted in prior studies which are
widely used when evaluating the performance of bug localization tools that
rely on information retrieval techniques [19, 53, 21, 5].

##  VII Related Works

In this section, we present prior work related to bug localization approaches
and discuss how our approach differs from them.

A plethora of work proposed lexical retriever-based approaches to localizing
bugs. Such approaches use BM25 and TF-IDF techniques to measure the similarity
between a bug report and source code files [5, 6, 54, 55, 56, 57]. For
example, BugLocator [5] used TF-IDF to measure the similarity between the bug
report and source code files using a revised vector space model. Sisman et al.
[58] and Amalgam [6] used version history along with the VSM model to localize
bugs. Shi et al. [59] found that BM25 and its variants are better than TF-IDF,
VSM, and rVSM in the bug localization task. BoostNSift [55] used BM25 for
method-level bug localization. Our approach, BLAZE, differs from prior work
since it leverages the advantages of both the BM25-based retrieval method and
deep learning-based technique to perform cross-project bug localization.

Other work proposed deep learning-based approaches to localizing bugs [53, 43,
5, 60, 11]. For example, FLIM [53] used CodeBERT embedding for calculating the
similarity between bug reports and source code files at the function level.
DNNLoc [61] combined a deep neural network with the VSM model to increase
effectiveness across different types of metrics such as textual similarity,
class name similarity, bug-fixing recency, and frequency. NP-CNN [43] employed
a Convolutional Neural Network (CNN) to leverage both lexical and semantical
properties. BL-GAN [12] proposed a generative adversarial network-based
solution for bug localization. Several studies [42, 9, 8, 10] incorporated
program structure information in the model to generate more useful program
representation. For example, Liang et al. [10] and Li et al. [8] used abstract
syntax trees, data flow graphs, and program flow graphs to generate a better
representation of source code. While BLAZE employs an embedding model similar
to the techniques discussed previously, it differs by its ability to be
applied to new, unseen projects without requiring additional training.

Several studies offered test case coverage or spectrum-based solutions for bug
localization [62, 63, 64, 65, 66, 67, 68, 69]. For example, FusionLocalizer
[69] combines multiple test case-based fault localizers. However, their
approach depends on tests, whereas BLAZE uses an information retrieval
approach solely based on bug reports. Moreover, the selection step in the
FusionLocalizer technique may introduce additional bias, while BLAZE employs
Reciprocal Rank Fusion (RRF) to consider the ranks of all retrievers,
effectively reducing bias. Similarly, UniLoc [70] uses information retrieval
for localizing bugs in continuous integration tools, while BLAZE focuses on
bug reports. Additionally, CrossLocator [71] introduces a translation-based
method for projects that uses multiple languages for bug reports (e.g.,
English, Chinese), whereas BLAZE identifies bugs in projects involving
different programming languages.

While numerous studies have been conducted on bug localization tools, most of
them are project-context approaches, i.e., the research on cross-project bug
localization is limited in comparison. For example, TRANP-CNN [15] used a
combination of transfer learning and CNN models to attain competitive
performance in cross-project bug localization at the file level. CooBa [16]
follows a similar approach but employs a shared encoder to capture cross-
project features. Such approaches still require further training before
deploying and making them work for new projects. In contrast, BLAZE is
designed in a way that it can work on a new project without further training.
DSSDPP [72] investigated a sampling approach to address disparities between
source and target projects, whereas xLoc [73] used transformers to identify
bugs in cross-language API calls. Nevertheless, both tools concentrate on
method-level defect prediction, in contrast to BLAZE, which localizes bugs at
the file level.

Previous research has also explored embedding-based methods for bug
localization. SemanticCodeBERT [20] integrates semantic flow graph data, while
FBL-BERT [19] uses changeset data. However, neither focuses on hard examples
to improve feature separation. In contrast, BLAZE learns from hard examples to
significantly enhance model performance.

##  VIII Conclusion

In this paper, we introduce BLAZE, a bug localization tool that operates
without project-specific training, facilitating cross-project and cross-
language bug localization. BLAZE employs a dynamic chunking and hard example
learning methodology to model lengthy source code files. To support cross-
project and cross-language bug localizers, we curate BeetleBox—one of the
largest (if not _the_ largest) cross-project and cross-language bug
localization datasets. We evaluate BLAZE with 31,745 bug reports from 34 real-
world projects, showing up to a 100% improvement in the Top 1 metric against
cross-project localizers and up to a 60% improvement over modern large
language model approaches. Our results suggest that the use of dynamic
chunking and hard example learning is key in enhancing the performance of bug
localization tools.

## References

  * [1] T. Koomen and M. Pol, _Test Process Improvement: A Practical Step-by-Step Guide to Structured Testing_. USA: Addison-Wesley Longman Publishing Co., Inc., 1999.
  * [2] T. D. LaToza and B. A. Myers, “Developers ask reachability questions,” in _Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - ICSE '10_. ACM Press, 2010.
  * [3] G. Tassey, “The economic impacts of inadequate infrastructure for software testing,” 2002.
  * [4] A. El-Deeb, “What the software industry is measuring?” _ACM SIGSOFT Software Engineering Notes_ , vol. 48, no. 2, pp. 10–11, Apr. 2023.
  * [5] J. Zhou, H. Zhang, and D. Lo, “Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports,” in _2012 34th International Conference on Software Engineering (ICSE)_. IEEE, Jun. 2012.
  * [6] S. Wang and D. Lo, “Version history, similar report, and structure: putting them together for improved bug localization,” in _Proceedings of the 22nd International Conference on Program Comprehension_. ACM, Jun. 2014.
  * [7] M. Fischer, M. Pinzger, and H. Gall, “Analyzing and relating bug report data for feature tracking,” in _10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings._ IEEE, 2003.
  * [8] Y. Li, S. Wang, T. N. Nguyen, and S. V. Nguyen, “Improving bug detection via context-based code representation learning and attention-based neural networks,” _Proceedings of the ACM on Programming Languages_ , vol. 3, no. OOPSLA, pp. 1–30, Oct. 2019.
  * [9] Y. Xiao, J. Keung, K. E. Bennin, and Q. Mi, “Improving bug localization with word embedding and enhanced convolutional neural networks,” _Information and Software Technology_ , vol. 105, pp. 17–29, Jan. 2019.
  * [10] H. Liang, L. Sun, M. Wang, and Y. Yang, “Deep learning with customized abstract syntax tree for bug localization,” _IEEE Access_ , vol. 7, pp. 116 309–116 320, 2019.
  * [11] J. Zhang, R. Xie, W. Ye, Y. Zhang, and S. Zhang, “Exploiting code knowledge graph for bug localization via bi-directional attention,” in _Proceedings of the 28th International Conference on Program Comprehension_. ACM, Jul. 2020.
  * [12] Z. Zhu, H. Tong, Y. Wang, and Y. Li, “BL-GAN: Semi-supervised bug localization via generative adversarial network,” _IEEE Transactions on Knowledge and Data Engineering_ , pp. 1–14, 2022.
  * [13] P. S. Kochhar, Y. Tian, and D. Lo, “Potential biases in bug localization,” in _Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering_. ACM, Sep. 2014.
  * [14] A. Sainani, P. R. Anish, V. Joshi, and S. Ghaisas, “Extracting and classifying requirements from software engineering contracts,” in _2020 IEEE 28th International Requirements Engineering Conference (RE)_. IEEE, Aug. 2020.
  * [15] X. Huo, F. Thung, M. Li, D. Lo, and S.-T. Shi, “Deep transfer bug localization,” _IEEE Transactions on Software Engineering_ , vol. 47, no. 7, pp. 1368–1380, Jul. 2021.
  * [16] Z. Zhu, Y. Li, H. Tong, and Y. Wang, “CooBa: Cross-project bug localization via adversarial transfer learning,” in _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence_. International Joint Conferences on Artificial Intelligence Organization, Jul. 2020.
  * [17] Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, “CodeBERT: A pre-trained model for programming and natural languages,” in _Findings of the Association for Computational Linguistics: EMNLP 2020_. Online: Association for Computational Linguistics, Nov. 2020, pp. 1536–1547.
  * [18] D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou, “Graphcodebert: Pre-training code representations with data flow,” 2021.
  * [19] A. Ciborowska and K. Damevski, “Fast changeset-based bug localization with BERT,” in _Proceedings of the 44th International Conference on Software Engineering_. ACM, May 2022\.
  * [20] Y. Du and Z. Yu, “Pre-training code representation with semantic flow graph for effective bug localization,” in _Proceedings of the 31th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , ser. ESEC/FSE 2023, 2023.
  * [21] X. Ye, R. Bunescu, and C. Liu, “Learning to rank relevant files for bug reports using domain knowledge,” in _Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering - FSE 2014_. ACM Press, 2014.
  * [22] Z. Qiao, W. Ye, D. Yu, T. Mo, W. Li, and S. Zhang, “Improving knowledge graph completion with generative hard negative mining,” in _Findings of the Association for Computational Linguistics: ACL 2023_ , A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 5866–5878.
  * [23] H. Xuan, A. Stylianou, X. Liu, and R. Pless, _Hard Negative Examples are Hard, but Useful_. Springer International Publishing, 2020, p. 126–142.
  * [24] C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan, “SWE-bench: Can language models resolve real-world github issues?” in _The Twelfth International Conference on Learning Representations_ , 2024.
  * [25] A. Neelakantan, T. Xu, R. Puri, A. Radford, J. M. Han, J. Tworek, Q. Yuan, N. Tezak, J. W. Kim, C. Hallacy, J. Heidecke, P. Shyam, B. Power, T. E. Nekoul, G. Sastry, G. Krueger, D. Schnurr, F. P. Such, K. Hsu, M. Thompson, T. Khan, T. Sherbakov, J. Jang, P. Welinder, and L. Weng, “Text and code embeddings by contrastive pre-training,” 2022.
  * [26] Anonymous, “Blaze: Bug localization in zero-shot environment,” _Zenodo_ , 2023. [Online]. Available: https://zenodo.org/record/7897697
  * [27] C. Liu, X. Xia, D. Lo, Z. Liu, A. E. Hassan, and S. Li, “CodeMatcher: Searching code based on sequential semantics of important query words,” _ACM Transactions on Software Engineering and Methodology_ , vol. 31, no. 1, pp. 1–37, Sep. 2021.
  * [28] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186.
  * [29] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” 2019.
  * [30] I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The long-document transformer,” 2020.
  * [31] D. Zhang*, W. Ahmad*, M. Tan, H. Ding, R. Nallapati, D. Roth, X. Ma, and B. Xiang, “Codesage: Code representation learning at scale,” in _The Twelfth International Conference on Learning Representations_ , 2024.
  * [32] J. Wang, X. Yi, R. Guo, H. Jin, P. Xu, S. Li, X. Wang, X. Guo, C. Li, X. Xu, K. Yu, Y. Yuan, Y. Zou, J. Long, Y. Cai, Z. Li, Z. Zhang, Y. Mo, J. Gu, R. Jiang, Y. Wei, and C. Xie, “Milvus: A purpose-built vector data management system,” in _Proceedings of the 2021 International Conference on Management of Data_ , ser. SIGMOD/PODS ’21. ACM, Jun. 2021.
  * [33] C. Gormley and Z. Tong, _Elasticsearch: The Definitive Guide_ , 1st ed. O’Reilly Media, Inc., 2015.
  * [34] S. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford, “Okapi at trec-3,” in _Overview of the Third Text REtrieval Conference (TREC-3)_. Gaithersburg, MD: NIST, January 1995, pp. 109–126.
  * [35] S. Sachdev, H. Li, S. Luan, S. Kim, K. Sen, and S. Chandra, “Retrieval on source code: a neural code search,” in _Proceedings of the 2nd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages_. ACM, Jun. 2018.
  * [36] Z. Abbasiantaeb and S. Momtazi, “Text-based question answering from information retrieval and deep neural network perspectives: A survey,” _WIREs Data Mining and Knowledge Discovery_ , vol. 11, no. 6, May 2021.
  * [37] Y. Luan, J. Eisenstein, K. Toutanova, and M. Collins, “Sparse, dense, and attentional representations for text retrieval,” _Transactions of the Association for Computational Linguistics_ , vol. 9, pp. 329–345, 2021.
  * [38] S. Wang, S. Zhuang, and G. Zuccon, “BERT-based dense retrievers require interpolation with BM25 for effective passage retrieval,” in _Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval_. ACM, Jul. 2021.
  * [39] G. V. Cormack, C. L. A. Clarke, and S. Buettcher, “Reciprocal rank fusion outperforms condorcet and individual rank learning methods,” in _Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval_. ACM, jul 2009.
  * [40] M. Bendersky, H. Zhuang, J. Ma, S. Han, K. Hall, and R. McDonald, “Rrf102: Meeting the trec-covid challenge with a 100+ runs ensemble,” 2020.
  * [41] T. Chen, M. Zhang, J. Lu, M. Bendersky, and M. Najork, “Out-of-domain semantics to the rescue! zero-shot hybrid retrieval models,” in _Lecture Notes in Computer Science_. Springer International Publishing, 2022, pp. 95–110.
  * [42] Y. Xiao, J. Keung, K. E. Bennin, and Q. Mi, “Machine translation-based bug localization technique for bridging lexical gap,” _Information and Software Technology_ , vol. 99, p. 58–61, Jul. 2018.
  * [43] X. Huo, M. Li, and Z.-H. Zhou, “Learning unified features from natural and programming languages for locating buggy source code,” in _IJCAI_ , 2016\.
  * [44] L. Moreno, J. J. Treadway, A. Marcus, and W. Shen, “On the use of stack traces to improve text retrieval-based bug localization,” in _2014 IEEE International Conference on Software Maintenance and Evolution_. IEEE, Sep. 2014.
  * [45] B. Dit, M. Revelle, M. Gethers, and D. Poshyvanyk, “Feature location in source code: a taxonomy and survey,” _Journal of Software: Evolution and Process_ , vol. 25, no. 1, p. 53–95, Nov. 2011.
  * [46] J. Lee, D. Kim, T. F. Bissyandé, W. Jung, and Y. Le Traon, “Bench4bl: reproducibility study on the performance of ir-based bug localization,” in _Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis_ , ser. ISSTA ’18. ACM, Jul. 2018.
  * [47] M. Kim and E. Lee, “Are datasets for information retrieval-based bug localization techniques trustworthy?: Impact analysis of bug types on irbl,” _Empirical Software Engineering_ , vol. 26, no. 3, Mar. 2021.
  * [48] G. Fan, S. Chen, C. Gao, J. Xiao, T. Zhang, and Z. Feng, “Rapid: Zero-shot domain adaptation for code search with pre-trained models,” _ACM Transactions on Software Engineering and Methodology_ , vol. 33, no. 5, p. 1–35, Jun. 2024.
  * [49] L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” _Journal of machine learning research_ , vol. 9, no. 11, 2008.
  * [50] N. Ratner, Y. Levine, Y. Belinkov, O. Ram, I. Magar, O. Abend, E. Karpas, A. Shashua, K. Leyton-Brown, and Y. Shoham, “Parallel context windows for large language models,” in _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 6383–6402.
  * [51] M. Ivgi, U. Shaham, and J. Berant, “Efficient Long-Text Understanding with Short-Text Models,” _Transactions of the Association for Computational Linguistics_ , vol. 11, pp. 284–299, 03 2023.
  * [52] O. Khattab and M. Zaharia, “ColBERT,” in _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. ACM, Jul. 2020\.
  * [53] H. Liang, D. Hang, and X. Li, “Modeling function-level interactions for file-level bug localization,” _Empirical Software Engineering_ , vol. 27, no. 7, Oct. 2022.
  * [54] V. Murali, L. Gross, R. Qian, and S. Chandra, “Industry-scale IR-based bug localization: A perspective from facebook,” in _2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)_. IEEE, May 2021\.
  * [55] A. Razzaq, J. Buckley, J. V. Patten, M. Chochlov, and A. R. Sai, “BoostNSift: A query boosting and code sifting technique for method level bug localization,” in _2021 IEEE 21st International Working Conference on Source Code Analysis and Manipulation (SCAM)_. IEEE, Sep. 2021.
  * [56] F. Niu, C. Mayr-Dorn, W. K. G. Assunção, L. Huang, J. Ge, B. Luo, and A. Egyed, “The ablots approach for bug localization: is it replicable and generalizable?” in _2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR)_. IEEE, May 2023.
  * [57] R. Almhana, M. Kessentini, and W. Mkaouer, “Method-level bug localization using hybrid multi-objective search,” _Information and Software Technology_ , vol. 131, p. 106474, 2021.
  * [58] B. Sisman and A. C. Kak, “Incorporating version histories in information retrieval based bug localization,” in _2012 9th IEEE Working Conference on Mining Software Repositories (MSR)_. IEEE, Jun. 2012.
  * [59] Z. Shi, J. Keung, and Q. Song, “An empirical study of BM25 and BM25f based feature location techniques,” in _Proceedings of the International Workshop on Innovative Software Development Methodologies and Practices_. ACM, Nov. 2014.
  * [60] Z. Zhu, Y. Li, Y. Wang, Y. Wang, and H. Tong, “A deep multimodal model for bug localization,” _Data Mining and Knowledge Discovery_ , Apr. 2021.
  * [61] A. N. Lam, A. T. Nguyen, H. A. Nguyen, and T. N. Nguyen, “Bug localization with combination of deep learning and information retrieval,” in _2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC)_. IEEE, May 2017.
  * [62] B. Vancsics, F. Horváth, A. Szatmári, and Á. Beszédes, “Fault localization using function call frequencies,” _Journal of Systems and Software_ , vol. 193, p. 111429, Nov. 2022.
  * [63] Y. Kim, S. Mun, S. Yoo, and M. Kim, “Precise learn-to-rank fault localization using dynamic and static features of target programs,” _ACM Transactions on Software Engineering and Methodology_ , vol. 28, no. 4, pp. 1–34, Oct. 2019.
  * [64] M. Wen, J. Chen, Y. Tian, R. Wu, D. Hao, S. Han, and S.-C. Cheung, “Historical spectrum based fault localization,” _IEEE Transactions on Software Engineering_ , vol. 47, no. 11, pp. 2348–2368, Nov. 2021.
  * [65] R. Widyasari, G. A. A. Prana, S. A. Haryono, S. Wang, and D. Lo, “Real world projects, real faults: evaluating spectrum based fault localization techniques on python projects,” _Empirical Software Engineering_ , vol. 27, no. 6, Aug. 2022.
  * [66] A. Z. H. Yang, C. Le Goues, R. Martins, and V. Hellendoorn, “Large language models for test-free fault localization,” in _Proceedings of the IEEE/ACM 46th International Conference on Software Engineering_ , ser. ICSE ’24. ACM, Feb. 2024.
  * [67] Z. Zhang, Y. Lei, X. Mao, M. Yan, X. Xia, and D. Lo, “Context-aware neural fault localization,” _IEEE Transactions on Software Engineering_ , vol. 49, no. 7, p. 3939–3954, Jul. 2023.
  * [68] A. Ghanbari, D.-G. Thomas, M. A. Arshad, and H. Rajan, “Mutation-based fault localization of deep neural networks,” in _2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, Sep. 2023.
  * [69] Lucia, D. Lo, and X. Xia, “Fusion fault localizers,” in _Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering_ , ser. ASE ’14. ACM, Sep. 2014.
  * [70] F. Hassan, N. Meng, and X. Wang, “Uniloc: Unified fault localization of continuous integration failures,” _ACM Transactions on Software Engineering and Methodology_ , vol. 32, no. 6, p. 1–31, Sep. 2023.
  * [71] X. Xia, D. Lo, X. Wang, C. Zhang, and X. Wang, “Cross-language bug localization,” in _Proceedings of the 22nd International Conference on Program Comprehension_ , ser. ICSE ’14. ACM, Jun. 2014.
  * [72] Z. Li, H. Zhang, X.-Y. Jing, J. Xie, M. Guo, and J. Ren, “DSSDPP: Data selection and sampling based domain programming predictor for cross-project defect prediction,” _IEEE Transactions on Software Engineering_ , vol. 49, no. 4, pp. 1941–1963, Apr. 2023.
  * [73] H. Yang, Y. Nong, T. Zhang, X. Luo, and H. Cai, “Learning to detect and localize multilingual bugs,” 2024.