# Beyond Benchmarking: A New Paradigm for Evaluation and Assessment of Large
Language Models

Jin Liu  Qingquan Li  Wenlong Du
Independent Researchers

###### Abstract

In current benchmarks for evaluating large language models (LLMs), there are
issues such as evaluation content restriction, untimely updates, and lack of
optimization guidance. In this paper, we propose a new paradigm for the
measurement of LLMs: Benchmarking-Evaluation-Assessment. Our paradigm shifts
the “location” of LLM evaluation from the “examination room” to the
“hospital”. Through conducting a “physical examination” on LLMs, it utilizes
specific task-solving as the evaluation content, performs deep attribution of
existing problems within LLMs, and provides recommendation for optimization.

##  1 Introduction

Currently, research on large language models (LLMs) has become a hot topic. In
order to measure the performance of LLMs, a large number of studies on
evaluation benchmarks of LLMs have been spawned. Those includes general
evaluation benchmarks such as SuperCLUE Xu et al. (2023) and MMBench Liu et
al. (2023), domain knowledge evaluation datasets such as Xiezhi Gu et al.
(2024) and FinEval Zhang et al. (2023), and safety and hallucination
evaluation methods such as SafetyPrompts Sun et al. (2023) and HaluEval Li et
al. (2023). Based on this, each LLM has a wealth of benchmarks to measure its
own performance. However, in the promotion of LLMs, we often see such titles:
“Surpassing GPT-4, we ranks first in the capability of …” Many developers say
that their LLMs are the best in some certain abilities, which makes us re-
examine the current LLM benchmarking paradigm: Can a batch of static data and
a score be used to accurately and completely measure the performance of a LLM?

##  2 Potential Issues in Current Paradigm of Benchmarking LLMs

###  2.1 Limited Benchmarking Capability: Knowledge

Generative language models can finish given tasks through natural language
dialogue. Therefore, most benchmarks of LLMs adopt the common evaluation form
in current human society - exams, that is, the knowledge to be evaluated is
written into questions for LLMs to answer. For instance, a SuperCLUE’s example
of evaluation on semantics understanding ability of LLMs is as follows:

  * •

Question: In the middle of the party, a Chinese man said to a foreigner: “I’m
going to ‘Fangbian’.” What does “Fangbian” mean here?

  * •

Assistant A’s Answer: In this context, “Fangbian” means to go to the bathroom
or toilet. The Chinese man say this to politely tell the foreigner that he
need to leave for a while to go to the bathroom. This is a euphemism to avoid
directly mentioning the topic of going to the bathroom.

  * •

Assistant B’s Answer: In this context, “Fangbian” means that the Chinese man
need to go to the toilet or take care of other personal needs and need to
leave for a while. This is a common polite expression to indicate that he is
leaving temporarily without affecting the party.

Especially in the benchmarks of evaluating professional LLMs, many benchmarks
directly use professional qualification examinations, such qualification exam
for legal professionals, certification of securities professional, and medical
licensing examination to evaluate the ability of a certain professional field
of LLMs. An objective score can be given through exams. However, just like
students who graduate will eventually enter the workplace, LLMs also will be
applied in a certain scenario. Can the exam truly evaluate the ability of LLMs
to solve professional problems? For example, how to evaluate whether the
contract written by LLMs is usable? How to evaluate whether the investment
advice by LLMs is reasonable? The knowledge evaluation benchmark based on
exams for LLMs needs to be upgraded to evaluate the ability of LLMs to solve
certain professional problems.

###  2.2 Evaluation Datasets Lack Dynamic Updating

Information in the real world is constantly changing. Based on this,
developers will also update the training set to put new knowledge into LLMs.
Some existing benchmarks are periodically upgraded, but most evaluation
datasets have not been updated after being released. However, in many
scenarios, new information is constantly pouring in, which required that the
upgrade speed of evaluation on LLMs must keep up with the update of
information. For example, in security scenarios, new sensitive events may
occur everyday. It is necessary to update the security evaluation data timely
to measure whether the LLM will generate unsafe responses. If the safety red
line is crossed, the application of LLMs will be blown. Current dynamic update
methods include: crawling of public data, which cannot guarantee the
comprehensivceness of updates; feedback from online problems, which is a post-
hoc method. Therefore, it is necessary to discover the problems of LLMs in
advance and comprehensively through dynamically updated evaluations, so as to
reduce online problems and improve user experience.

###  2.3 Inadequacy of Evaluation Metrics for Guiding Model Optimization

For wrong questions in the exam, students often record the wrong questions,
learn the corresponding knowledge, and do similar questions to fill in the
gaps. The purpose of doing this is to avoid the same type of mistakes in the
next exam. However, for LLMs, the goal of evaluation should not be to improve
the performance on a certain evaluation metric, but to dis cover the problems
of LLMs in different dimensions, and the improve the ability to solve real
problems. Moreover, there is also the risk of data leakage when training LLMs
on exam questions. Many existing evaluation benchmarks only generate a
capability score, which makes it difficult ot provide targeted guidance on
which abilities the LLM should improve.

##  3 Benckmarking-Evaluation-Assessment: A New Paradigm for Measuring the
Capability Level of LLMs

In view of the above problems, we can evaluate the LLMs dynamically and beyond
knowledge, and attribute the problems of LLMs in a targeted way, which can be
compared to the process of a man discovering health problems and receiving
diagnoses through physical examinations in hospitals:

  1. 1.

First, he undergo a comprehensive physical examination. The content of the
physical examination covers most organs of his body, but the granularity of
the examination is relatively coarse. Here we assume that his blood pressure
is high.

  2. 2.

Secondly, in response to the blood pressure problem of the examinee, the
hospital prescribes more targeted examinations, including 24-hour dynamic
blood pressure, four items of standing blood pressure, electrocardiogram,
cardiac color ultrasound, etc., to explore the reasons for the high blood
pressure of the examinee through fine-grained metrics.

  3. 3.

Combined with fine-grained evaluation metrics, doctors diagnose the cause of
the patient’s illness (mental stress, primary hypertension, secondary
hypertension, etc.) and provide treatment plans.

Referring to the medical treatment process, the ability level of LLMs can also
be measured in three progressive stages: benchmarking-evaluation-assessment.

  1. 1.

Benchmarking: A comprehensive and coarse-grained score is conducted for the
LLM to find out the lack of ability of the LLM.

  2. 2.

Evaluation: Based on the lacked capabilities, the LLM is evaluated by
finishing professional tasks to further explore the specific problems of the
LLM in this capability.

  3. 3.

Assessment: The problem attribution of the LLM in this capability is given
through a “doctor model” combined with the fine-grained metrics, to providing
direction for th ability improvement of the LLM.

Fig. 1 shows the comparison of measurement on human health and LLM’s ability.

Figure 1: The comparison of measurement on human health and LLM’s ability.

There is a progressive relationship between benchmarking, evaluation, and
assessment. Benchmarking is only a basic check of the LLM. It uses an
objective score to find problems in a certain capability of the LLM, but does
not draw conclusions. Evaluation is the core of our paradigm. Just as the
doctor can prescribe the most targeted examinations for patients based on
their knowledge and experience, evaluation is a way to deeply measure the
level of a LLM on a certain capability. Through step-by-step and hierarchical
measurement of different tasks, evaluation explores the capabilities of LLMs
in a fine-grained manner. Assessment, as the last stage of out paradigm, is to
conduct a analysis of the results of the benchmarking and evaluation to draw
conclusions. Specifically, it is mainly to attribute the problems existing in
the capabilities of LLMs and provide guidance for optimization and solutions,
similar to the process of doctors prescribing medicine. Assessment is mainly a
data engineering task, which can also be executed by AI models. Current works
such as PandaLM Wang et al. (2024) and CritiqueLLM Ke et al. (2023) are
exploring the capabilities of LLMs in assessment. Fig. 2 shows the preliminary
architecture of our paradigm. Each part of it needs to be designed in depth
according to different scenarios and tasks. Currently, we provide a high-level
framework.

Figure 2: The architecture of our benchmarking-evaluation-assessment paradigm.

Our proposed LLM measurement paradigm benchmarking-evaluation-assessment can
alleviate the potential problems or current LLM evaluation benckmarks, as
follows:

  1. 1.

The process from benchmarking to evaluation is an expansion from traditional
knowledge exam to the evaluation of task-solving ability, breaking the
limitation of current LLM benchmarks being confined to knowledge.

  2. 2.

The iteration cycle of knowledge is often long, while the evaluation based on
task-solving is more conducive to the dynamic update of measuring the
capability of LLMs through the transformation of scenarios and tasks.

  3. 3.

In the assessment of LLMs, a doctor-like role is introduced to attribute the
problems of the LLM on a specific capability, thereby providing optimization
direction for the LLM on that capability.

Visually, the above paradigm upgrade can be seen as the transformation from
the LLM entering the “examination room” to entering the “hospital”. The former
uses a test paper to measure the LLM’s mastery of knowledge, while the
feedback is only to the supplement of knowledge. The latter locates the
shortcomings of the LLM through a comprehensive physical examination, explores
the roots causes of the LLM’s lack of capabilities, and provides a diagnosis
and treatment plan for the LLM through doctor models, which can truly discover
and “cure” the LLM’s problems.

##  4 Conclusion

As mentioned above, the current “exam paper” for LLMs is knowledge-based
evaluation. When the measurement paradigm of LLMs changes from “examination
room” to “hospital”, how to issue a “checklist” for LLMs, that is, what
dimensions of LLMs should be evaluated, is an important topic. The LLM is not
just a knowledge center like a search engine, but a tool to improve
productivity. Therefore, the ability of a LLM to solve specific tasks should
be the goal of LLM measurement. It is necessary to decompose the task-solving
process of the LLM as the evaluation dimension. For example, to what extent
does the LLM master knowledge? How does the LLM internalize extrenal
knowledge? How does the LLM solve tasks such as language understanding,
logical reasoning, and text generation step by step? Those capability
dimensions will serve as a design guide for evaluation dimensions and metrics.
We will demonstrate how to split the capability dimension of the LLM to design
the evaluation framework. Furthermore, for the assessment, how to mine and
attribute problems of the LLM automatically through different methods such as
data engineering and models is the last step of giving a “checklist”.

## References

  * Gu et al. (2024) Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye, Lin Zhang, Jianchen Wang, Yixin Zhu, Sihang Jiang, Zhuozhi Xiong, Zihan Li, Weijie Wu, et al. 2024.  Xiezhi: An ever-updating benchmark for holistic domain knowledge evaluation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 38, pages 18099–18107.
  * Ke et al. (2023) Pei Ke, Bosi Wen, Zhuoer Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, et al. 2023.  Critiquellm: Scaling llm-as-critic for effective and explainable evaluation of large language model generation.  _arXiv preprint arXiv:2311.18702_.
  * Li et al. (2023) Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023.  Halueval: A large-scale hallucination evaluation benchmark for large language models.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 6449–6464.
  * Liu et al. (2023) Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. 2023.  Mmbench: Is your multi-modal model an all-around player?  _arXiv preprint arXiv:2307.06281_.
  * Sun et al. (2023) Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, and Minlie Huang. 2023.  Safety assessment of chinese large language models.  _arXiv preprint arXiv:2304.10436_.
  * Wang et al. (2024) Yidong Wang, Zhuohao Yu, Wenjin Yao, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, et al. 2024.  Pandalm: An automatic evaluation benchmark for llm instruction tuning optimization.  In _The Twelfth International Conference on Learning Representations_.
  * Xu et al. (2023) Liang Xu, Anqi Li, Lei Zhu, Hang Xue, Changtai Zhu, Kangkang Zhao, Haonan He, Xuanwei Zhang, Qiyue Kang, and Zhenzhong Lan. 2023.  Superclue: A comprehensive chinese large language model benchmark.  _arXiv preprint arXiv:2307.15020_.
  * Zhang et al. (2023) Liwen Zhang, Weige Cai, Zhaowei Liu, Zhi Yang, Wei Dai, Yujie Liao, Qianru Qin, Yifei Li, Xingyu Liu, Zhiqiang Liu, et al. 2023.  Fineval: A chinese financial domain knowledge evaluation benchmark for large language models.  _arXiv preprint arXiv:2308.09975_.


processing times for $s\in[1,5]$ by $\frac{16.48}{7.6}$, the average
difference between the rescaled time and time for $s\in[1,25]$ was less than
$10\%$. The different number of guesses can be explained by the fact that the
maximum size of pool of guesses has size bounded between $n$ (all the machines
have equal speeds) and $nm$ (all the machines have different speeds). This
means that, to make the algorithm more practical, a more approximation
oriented approach shall be considered. For example, if the ratio between $LB$
(lower bound on optimal $C_{\max}$) and $UB$ (upper bound on optimal
$C_{\max}$) is less than some, given by an user, $1+\varepsilon$, then perhaps
the calculations can be ended a bit earlier.

$m$ | speeds | $b_{\min}$ |  Time of computations [$\mu$s] | $\\#Cut\textrm{-}Vertices$ | $\\#Assignments$ |  $\\#Flows$ tested | $\frac{\\#\texttt{YES}}{\\#\texttt{NO}}$
---|---|---|---|---|---|---|---
$4$ | $5,5,5,1$ | $13$ | $6795740.1$ | $8.4$ | $2605936.8$ | $39373.6$ | $1.33$
$6$ | $5,5,5,5,1,1$ | $8$ | $42390182.0$ | $6.3$ | $1764089.8$ | $238880.8$ | $0.7$
$8$ | $5,5,5,5,5,2,1,1$ | $6$ | $26534304.7$ | $4.9$ | $320237.0$ | $116314.0$ | $1.59$
Table 13:  A more detailed statistics for a sample data $n=40$, $b_{\min}$.
The statistics were calculated for $100$ instances.

For $b_{\textrm{avg}}$ and $b_{\max}$, when we increase number of machines,
the processing times also increase; cf. Table 11 and Table 11. However, for
$b_{\min}$, it is not the case, see Table 13. Observe that for $b_{\min}$
increasing the number of machines, decreases the number of blocks that has to
be formed. Hence the algorithm tries up to $4^{8.4}$, $6^{6.3}$, $8^{4.9}$
distinct assignments, respectively; $4^{8.4}>6^{6.3}>8^{4.9}$. Another fact is
that when we increase the number of machines, with constant bound on speeds
and fixed number of vertices, the size of the set of $C_{\max}$ candidates
becomes stable; however, the YESanswer becomes more likely. By YESanswer, we
mean that for a given guess of $C_{\max}$, there exists a schedule. There is
fundamental difference between YESand NOanswer with respect to computations
times. To answer YES, it is enough to find one valid assignment of cut-
vertices and a schedule consistent with the assignment; to answer NO, we have
to check all the assignments. Also due to this, for $b_{\min}$, the number of
assignments checked should decrease when the number of machines increase.

However, the assignments of cut vertices are verified with respect to guessed
$C_{\max}$ and incompatibility relation and only valid ones are used. Hence,
by increasing the number of machines, the number of _valid_ assignments should
increase. On the other hand, due to more more prevalent YESanswers, the number
of valid assignments checked could become lower. Interestingly, the maximum
average number of valid assignments checked was for $m=6$, which also
corresponds to maximum average procesing time, and to minimum ratio of YESto
NOanswers. We think that a deeper theoretical analysis and more measurements
are needed to characterize the behaviour of the algorithm. In particular an
analysis how much more costly in time terms is NOanswer comparing to
YESanswer. If a difference is huge, then perhaps instead of bisection it would
be better to from some point iterate over candidates for $C_{\max}$ starting
from the biggest one.

In our example, for $m=4$, $m=6$, $m=8$, on average $1.5\%$, $13.5\%$,
$36.3\%$ of assignments were valid, respectively. Hence, a proposed heuristics
is to use greedy assignment / branch-and-bound algorithm to filter out more
assignment of cut-vertices. A assignment of even some cut-vertices gives a
lower bound on the length of any schedule consistent with this assignment.
Hence, if it not better than the best $C_{\max}$ found so far, there is no
point in even considering this assignment any further. Moreover, the
assignment has to be consistent with the incompatibility relation. There is
not point to continue generating the assignments if a part of the assignment
is not compatible with the relation.

Finally, for the measured instances, Boykov-Kolmogorov was consistently
faster, by between 10% and 20%, cf. Table 11 and Table 12

##  6 Open problems

Although we have explored the status of scheduling problems with conflict
block graphs to some extent, there are still some related problems remaining
to be solved. For example, we are interested in decreasing an approximation
factor of an algorithm for $P|G=\textit{block graph}|C_{\max}$ or proving some
inapproximability results. It is also interesting to know whether there exists
a PTAS for $Q|G=\textit{block graph},p_{j}=1|C_{\max}$ or any constant factor
approximation algorithm for $Q|G=\textit{block graph}|C_{\max}$. Other results
in the area that is the subject of this paper would be also very welcomed.

## References

  * Ausiello et al., (1999) Ausiello, G., Marchetti-Spaccamela, A., Crescenzi, P., Gambosi, G., Protasi, M., and Kann, V. (1999).  Complexity and Approximation: Combinatorial Optimization Problems and Their Approximability Properties.  Springer.
  * Baker and Coffman Jr., (1996) Baker, B. and Coffman Jr., E. (1996).  Mutual exclusion scheduling.  Theoretical Computer Science, 162(2):225–243.
  * Bodlaender and Jansen, (1993) Bodlaender, H. and Jansen, K. (1993).  On the complexity of scheduling incompatible jobs with unit-times.  In International Symposium on Mathematical Foundations of Computer Science, pages 291–300. Springer.
  * Bodlaender et al., (1994) Bodlaender, H., Jansen, K., and Woeginger, G. (1994).  Scheduling with incompatible jobs.  Discrete Applied Mathematics, 55(3):219–232.
  * Brucker, (2006) Brucker, P. (2006).  Scheduling Algorithms.  Springer.
  * Cygan et al., (2015) Cygan, M., Fomin, F., Kowalik, Ł., Lokshtanov, D., Marx, D., Pilipczuk, M., Pilipczuk, M., and Saurabh, S. (2015).  Parameterized algorithms.  Springer.
  * Das and Wiese, (2017) Das, S. and Wiese, A. (2017).  On minimizing the makespan when some jobs cannot be assigned on the same machine.  In 25th Annual European Symposium on Algorithms (ESA 2017). Schloss Dagstuhl-Leibniz-Zentrum für Informatik.
  * Dessouky et al., (1990) Dessouky, M., Lageweg, B., Lenstra, J. K., and van de Velde, S. (1990).  Scheduling identical jobs on uniform parallel machines.  Statistica Neerlandica, 44(3):115–123.
  * Diestel, (2005) Diestel, R. (2005).  Graph Theory, volume 173 of Graduate Texts in Mathematics.  Springer.
  * Epstein and Sgall, (2004) Epstein, L. and Sgall, J. (2004).  Approximation schemes for scheduling on uniformly related and identical parallel machines.  Algorithmica, 39(1):43–57.
  * Furmańczyk and Mkrtchyan, (2022) Furmańczyk, H. and Mkrtchyan, V. (2022).  Graph theoretic and algorithmic aspect of the equitable coloring problem in block graphs.  Discrete Mathematics and Theoretical Computer Science, 23(2).
  * Garey and Johnson, (1979) Garey, M. and Johnson, D. (1979).  Computers and Intractability: A Guide to the Theory of NP-Completeness.  W. H. Freeman.
  * Goldberg, (1985) Goldberg, A. (1985).  A new max-flow algorithm.  MIT/LCS/TM291, 545(1):1–12.
  * Goldberg et al., (2011) Goldberg, A. V., Hed, S., Kaplan, H., Tarjan, R. E., and Werneck, R. F. F. (2011).  Maximum flows by incremental breadth-first search.  In Demetrescu, C. and Halldórsson, M. M., editors, Algorithms - ESA 2011 - 19th Annual European Symposium, Saarbrücken, Germany, September 5-9, 2011. Proceedings, volume 6942 of Lecture Notes in Computer Science, pages 457–468. Springer.
  * Gomes et al., (2019) Gomes, G. d., Lima, C., and Santos, V. (2019).  Parameterized complexity of equitable coloring.  Discrete Mathematics and Theoretical Computer Science, 21(1):1–11.
  * Grage et al., (2019) Grage, K., Jansen, K., and Klein, K.-M. (2019).  An EPTAS for machine scheduling with bag-constraints.  In The 31st ACM Symposium on Parallelism in Algorithms and Architectures, pages 135–144.
  * Graham, (1966) Graham, R. (1966).  Bounds for certain multiprocessing anomalies.  Bell System Technical Journal, 45(9):1563–1581.
  * Harary, (1963) Harary, F. (1963).  A characterization of block-graphs.  Canadian Mathematical Bulletin, 6(1):1–6.
  * Hochbaum and Shmoys, (1988) Hochbaum, D. and Shmoys, D. (1988).  A polynomial approximation scheme for scheduling on uniform processors: Using the dual approximation approach.  SIAM Journal of Computing, 17(3):539–551.
  * Horowitz and Sahni, (1976) Horowitz, E. and Sahni, S. (1976).  Exact and approximate algorithms for scheduling nonidentical processors.  Journal of the ACM, 23(2):317–327.
  * Jansen et al., (2021) Jansen, K., Lassota, A., Maack, M., and Pikies, T. (2021).  Total completion time minimization for scheduling with incompatibility cliques.  In Proceedings of the International Conference on Automated Planning and Scheduling, volume 31, pages 192–200.
  * Jansen and Maack, (2019) Jansen, K. and Maack, M. (2019).  An EPTAS for scheduling on unrelated machines of few different types.  Algorithmica, 81(10):4134–4164.
  * Kones and Levin, (2019) Kones, I. and Levin, A. (2019).  A unified framework for designing EPTAS for load balancing on parallel machines.  Algorithmica, 81(7):3025–3046.
  * Kowalczyk and Roel, (2017) Kowalczyk, D. and Roel, L. (2017).  An exact algorithm for parallel machine scheduling with conflicts.  Journal of Scheduling, 20(4):355–372.
  * Lawler et al., (1982) Lawler, E., Lenstra, J. K., and Kan, A. R. (1982).  Recent developments in deterministic sequencing and scheduling: A survey.  In Dempster, M., Lenstra, J. K., and Kan, A. R., editors, Deterministic and Stochastic Scheduling, volume 84 of NATO Advanced Study Institutes Series $($Series C – Mathematical and Physical Sciences$)$, pages 35–73. Springer.
  * Lenstra et al., (1990) Lenstra, J. K., Shmoys, D., and Tardos, É. (1990).  Approximation algorithms for scheduling unrelated parallel machines.  Mathematical Programming, 46(1-3):259–271.
  * Mallek et al., (2019) Mallek, A., Bendraouche, M., and Boudhar, M. (2019).  Scheduling identical jobs on uniform machines with a conflict graph.  Computers & Operations Research, 111:357–366.
  * Mallek and Boudhar, (2022) Mallek, A. and Boudhar, M. (2022).  Scheduling on uniform machines with a conflict graph: complexity and resolution.  International Transactions in Operational Research.
  * Nijenhuis and Wilf, (1978) Nijenhuis, A. and Wilf, H. (1978).  Combinatorial algorithms: for computers and calculators.  Academic Press.
  * Orlin, (2013) Orlin, J. (2013).  Max flows in $O(nm)$ time, or better.  In Proceedings of the 45th Annual ACM Symposium on Theory of Computing, pages 765–774.
  * Page and Solis-Oba, (2020) Page, D. and Solis-Oba, R. (2020).  Makespan minimization on unrelated parallel machines with a few bags.  Theoretical Computer Science, 821:34–44.
  * Pikies and Furmańczyk, (2022) Pikies, T. and Furmańczyk, H. (2022).  Scheduling on uniform and unrelated machines with bipartite incompatibility graphs.  In 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 661–671. IEEE Computer Society.
  * Pikies and Turowski, (2024) Pikies, T. and Turowski, K. (2024).  Block scheduling.  https://bitbucket.org/tpikies/block-scheduling/.
  * Pikies et al., (2022) Pikies, T., Turowski, K., and Kubale, M. (2022).  Scheduling with complete multipartite incompatibility graph on parallel machines: Complexity and algorithms.  Artificial Intelligence, 309:103711.
  * Shchepin and Vakhania, (2005) Shchepin, E. and Vakhania, N. (2005).  An optimal rounding gives a better approximation for scheduling unrelated machines.  Operations Research Letters, 33(2):127–133.


# Anomalous microwave response in the dissipative regime of topological
superconducting devices based on Bi2Te2.3Se0.7

Vasily Stolyarov  Advanced Mesoscience and Nanotechnology Centre, Moscow
Institute of Physics and Technology, 141700 Dolgoprudny, Russia  Sergei Kozlov
Laboratoire de Physique et d’étude des Matériaux, ESPCI Paris, CNRS, PSL
University, 75005 Paris, France  Dmitry Yakovlev<EMAIL_ADDRESS>Laboratoire de Physique et d’étude des Matériaux, ESPCI Paris, CNRS, PSL
University, 75005 Paris, France  Nicolas Bergeal  Laboratoire de Physique et
d’étude des Matériaux, ESPCI Paris, CNRS, PSL University, 75005 Paris, France
Cheryl Feuillet-Palma  Laboratoire de Physique et d’étude des Matériaux, ESPCI
Paris, CNRS, PSL University, 75005 Paris, France  Olga Skryabina  Advanced
Mesoscience and Nanotechnology Centre, Moscow Institute of Physics and
Technology, 141700 Dolgoprudny, Russia  Dmitry Lvov  Department of Applied
Physics, Low Temperature Laboratory, Aalto University School of Science, PO
Box 13500, 00076 Aalto, Finland  Mikhail Kupriyanov  Skobeltsyn Institute of
Nuclear Physics, Lomonosov Moscow State University, 119991 Moscow, Russia
Alexander Golubov  Faculty of Science and Technology and MESA+ Institute for
Nanotechnology, University of Twente, 7500 AE Enschede, The Netherlands;
Dimitri Roditchev  Laboratoire de Physique et d’étude des Matériaux, ESPCI
Paris, CNRS, PSL University, 75005 Paris, France

###### Abstract

Superconducting proximity junctions based on topological insulators are widely
believed to harbor Majorana-like bound states. The latter serves as a paradigm
non-local topological quantum computation protocols. Nowadays, a search for
topological phases in different materials, perspective for a realization of
topological qubits, is one of the central efforts in quantum physics. It is
motivated, in particular, by recent observation of anomalous ac Josephson
effect, which being a signature of Majorana physics. Its manifestations, such
as a fractional Josephson frequency and the absence of the first (or several
odd in more rare cases), Shapiro steps, were reported for different materials.

Here we study Shapiro steps in Nb/Bi2Te2.3Se0.7/Nb junctions, based on
ultrasmall single crystals of a 3D topological insulator synthesized by a
physical vapor deposition (PVD) technique. We present evidence that our
junctions are ballistic. When subjected to microwave radiation, the junctions
exhibit Shapiro steps, but the first step is missing. Typically it is assumed
that the missing first step (MFS) effect cannot be observed in the presence of
quasiparticle poisoning due to suppression of the 4$\pi$-periodic component.
Our findings within the context of the RSJ-model of Josephson junction
dynamics show that such behaviour of samples corresponds to a specific
condition, requiring a minimum of 5% of the 4$\pi$-component for disappearance
of the first Shapiro step.

Shapiro step missing, Topilogical insulator, Superconductivity, Ballistic
transport, 4$\pi$-periodic component

††preprint: ?

##  I Introduction

Andreev Bound States (ABS) emerge in Superconductor/Normal
metal/Superconductor (SNS) Josephson junctions as localized solutions of the
Bogoliubov-De Gennes equations within the normal region [1]. In SNS junctions
with topological order in the normal part, Majorana Zero Modes (MZM) are
believed to appear [2, 3, 4, 5, 6]. Localized Majorana fermions encode
topological Andreev Bound States ($T$-ABS) which coherently transfer $1e$
charge and, correspondingly, their current-phase relationship (CPR) is
$4\pi$-periodic, at least according to the theory. Such periodicity is
possible since the topological superconductors can accept single electrons
onto a pair of Majorana modes, and such single electrons have zero energy,
unlike the usual quasiparticles (bogoliubons).

Figure 1: Experimental observation of supercurrent in Josephson junction based
on ultra small single crystal of Bi2Te2.3Se0.7. a Scanning electron microscopy
image of the Nb/Bi2Te2.3Se0.7/Nb Josephson junction. The superconducting Nb
leads (S-regions) are shown in blue color; the flake of Bi2Te2.3Se0.7 in the
N-region is colored light green. b Schematic view of the device along with the
external electrical circuit. DC bias current $I$ is applied and the voltage
drop $V$ across the junction is measured. c Hysteretic $V(I)$ characteristic
of the junction measured at $T=20\,$mK. Supercurrent at $V_{RF}=0$ $\mu$V is
observed with a critical current $I_{C}=0.46\,\mu$A and a retrapping current
$I_{R}=0.22\,\mu$A, respectively. The solid black line corresponds to the
hysteretic $V(I)$ curve measured at $f_{RF}=1.5\,$GHz and $V_{RF}=20$ $\mu$V
with Shapiro steps. d Critical $I_{C}$ and retrapping $I_{R}$ currents as
function of $T$. The ballistic fit was performed using the Eilenberger
equations. The retrapping current was fitted by t-RSJ model (see text for
details).

Distinct features of $T$-ABS originate from that fact that a pair of Majorana
real fermions correspond to a complex fermion level. This level defines
fermion parity $\sigma$ in the junction where, let us say, $\sigma=+1$ if the
level is occupied and $\sigma=-1$ if it is empty. Energies of $t$-ABS, given
by $\epsilon_{\sigma}(\varphi)=\sigma E(\varphi)$, are non-degenerate with
respect to $\sigma$ and the function $E(\varphi)$ is $4\pi$-periodic and odd.
Unlike ABS in trivial SNS junctions, the branches $\epsilon_{\sigma}(\varphi)$
and $\epsilon_{-\sigma}(\varphi)$ have topologically protected crossings at
zero energy in the phases of $\varphi=\pi+2\pi n,n\in\mathbb{Z}$ (the ground
state is degenerate at these points with respect to $\sigma$). Due to these
protected crossings the current-phase relationship acquires anomalous
$4\pi$-component, in addition to the conventional $2\pi$-periodic one. The
sign of the $4\pi$-component can be positive or negative depending on the sign
of the parity $\sigma$.

The $4\pi$-component can be observed experimentally only if the parity does
not change over the duration of the measurement. Otherwise, if the parity
changes frequently, the $4\pi$-component contribution is averaged to zero.
This explains why rapid dynamics measurements are preferable. Thus,
previously, the $4\pi$-Josephson effect was probed in the high-frequency
regime in which $\varphi$ grows in time and crosses the degeneracy points
faster than a parity relaxation occurs [7].

The initial experimental confirmation of the absence of the first Shapiro step
at a voltage $V_{\rm 1}=\frac{hf}{2e}$ (where $h$ represents Planck’s
constant, $e$ the electron charge, and $f$ indicating frequency) was
demonstrated in spin-orbit coupled nanowires [2, 3, 4, 8], as well as in
three-dimensional topological insulators [9]. Later, HgTe/CdTe-based junction
experiments unveiled the absence of the first nine odd steps [10] and the
fractional Josephson radiation frequency. The debate surrounds the anomalous
interpretation of the ac Josephson effect due to unbroken time-reversal
symmetry, challenging expectations for localized Majorana states in Ref. [10].

Absence of $n=\pm 1$ Shapiro step was shown in exfoliated topological
insulator Bi2Se3 [11] flakes and Dirac semimetals [12, 13]. In particular,
high-intensity drives exhibited residual supercurrent [11], confirming the
$4\pi$-periodic nature of the ac Josephson effect. The partial fractional
observation of the Josephson effect linked to Joule overheating, the decrease
in parity lifetime, was evident in hysteretic $V(I)$-curves.

To explain the Missing First Step (MFS) effect, the phase winding frequency,
$f_{Jt}=eV/h$, is assumed to be a half of the usual one, $f_{J}=2eV/h$, being
a consequence of coherent $1e$ transport. Therefore, the first microwave-
induced step should occur at a voltage $eV_{1t}/h=hf/e$, for a given radiation
frequency $f$, which is twice higher than the usual voltage for the first
step,$V_{1}=hf/2e$. Thus, it appears that the first step is missing, which is
the MFS effect [11, 5, 6, 9]. An observation of the absence of odd Shapiro
steps as well as fractional frequency of Josephson radiation is believed to be
one of possible routes to prove the existence of the Majorana zero modes [14,
15, 16].

Figure 2: Shapiro maps dV/dI for different RF drive frequencies. a, d, g and j
The drive frequencies $f_{RF}=$ 0.9, 1, 1.3, and 2 GHz are plotted. The maps
cover negative (switching) and positive (retrapping) polarities of the bias
current and demonstrate a hysteresis. The left-hand side of a given map for
$I<0$ corresponds to the switching current, and the I$>$0 segment corresponds
to the retrapping current. b, e, h and k $V(I)$-curves in the normalized unit
for voltage. The negative polarity shows the switching current, and the
positive polarity shows the retrapping current. For I$>$0, the even-odd effect
is observed. The panel on the right shows the dI/dV as a function of bias
current at different RF powers, giving a clearer view of the effect. The
second Shapiro step emerges at a lower radiation power than first step. The
first step is recovered at sufficiently large RF drive amplitudes.

Comparing to trivial SNS junctions, ballistic SNS junctions exhibit ABS
energies with a $4\pi$-periodic dependence [17] $\pm\Delta\cos(\phi/2)$, where
$(\phi/2)$, is the superconducting phase difference and $\Delta$ is the
induced energy gap. However, recent theory indicates a $2\pi$-periodic static
CPR [17] in this case.

The MFS effect could also originate from trivial ballistic junctions, posing a
challenge to differentiate between possibilities. The ubiquity of the MFS
effect includes observations in amorphous superconducting nanowires and
nanobridges [18]. First steps or few steps could be masked by junction
switching or retrapping currents. Junction bistability, characterized by
abrupt voltage jumps, can hinder first step visibility. Recovery of the first
step involves increasing microwave frequency or power.

Distinguishing topological and trivial effects involves a proposed method [11]
involving quasiparticle poisoning due to Joule heating. Quasiparticles cause
rapid changes in the Majorana Zero-Mode (MZM) contribution, averaging it to
zero. This explains why only the first step disappears while the third step
remains, as Joule heating creates high quasiparticle population. High-
population quasiparticles can occupy Majorana fermion modes, altering parity
according to theory [19]. Thus, changing current from high to zero should
retain the presence of the first step.

In this letter, we demonstrate the anomalous Josephson effect, namely the
absence of the first Shapiro step. The studied SNS junction involves two Nb
electrodes coupled through an ultrasmall single nanocrystallite of a 3D
topological insulator Bi2Te2.3Se0.7 [20], acting as the normal (N) region of
the SNS jucntion. We present Shapiro step maps as the differential conductance
$dI/dV$ plotted versus the DC current bias and the RF drive power, in the 0.5
to 3 GHz frequency range. Strongly hysteretic curves $V(I)$ and the $I_{C}$(T)
dependence are also presented. The $I_{C}$(T) function is well described by
the ballistic electron theory of the SNS junctions (see also ref[20]). To
describe the Shapiro steps and their dependence of the RF drive power, the
two-channel thermal resistively shunted junction (t-RSJ) model is adopted. Our
important conclusion is that the first step is missing even in the retrapping
branch of the $V(I)$ curve, i.e., when the Joule heating is quite significant.
The model we use to analyze the data includes (i) quasiparticle overheating
due to the Joule effect [11], and (ii) thermally activated poisoning of the
MBS. We demonstrate a good agreement of the results with this extended t-RSJ
model.

##  II  Experiment

The Bi2Te2.3Se0.7 nanocrystalline ingots used to make our junctions were
synthesized by Physical Vapor Deposition (PVD) method [21]. Our detailed XRD
analysis shows that the ingots contain only a single phase of
Bi2Te2.3Se0.7.The studied SNS junction is shown in Fig.1 (a). The measured
thickness of the flake is $t=20$ nm. The length of the N-region is $L=150$,
nm, and the widths of the Nb electrodes are $w=500$, nm. Several junctions of
different sizes were fabricated and studied in the experiments. Fig.1 (b)
shows a schematic view of the Josephson junction and the external electrical
circuit is presented. The device is measured in a regime of DC bias current
$I$ where the voltage drop $V$ across the junction is present; the RF drive is
applied through a microwave generator.

The normal state resistance of the junction at room temperature is
$R_{N}\approx 1.2$ k$\Omega$. The resistance shows a conventional metallic
behavior if the temperature $T$ is decreased. An abrupt drop in resistance was
detected at $T=8$, K attributed to the superconducting phase transition in the
Nb electrodes that form the junction. The further cooling down of the junction
shows a smooth decrease of the resistance in a domain from 5 to 1 K. The
latter is characteristic for the superconducting proximity effect induced in
the TI flake by the closely spaced Nb electrodes [20].

Fig. 1(c) illustrates a hysteric voltage-current $V(I)$ relation, where the
critical current ($I_{C}$) and the retrapping current ($I_{R}$) are not equal.
The red curve represents the increasing bias current and shows the switching
(critical) current, a jump-like transition from superconducting to normal
state. The blue curve corresponds to the opposite direction of the current
($I$) and represents the retrapping current. The black curve illustrates the
voltage-current characteristic under the influence of an RF signal. Fig. 1(d)
displays the temperature dependence of both critical and retrapping currents.
Hysteresis appears at temperatures below 0.7 K. Additionally, the absence of a
low-temperature saturation in $T_{c}(T)$ suggests ballistic transport [20],
which is further supported by fits generated using the ballistic-limit
Eilenberger equations and their solutions obtained by Galaktionov and Zaikin
[17] .

The presence of an RF antenna 1(b), which is not ideally matched and is
located close to the sample, leads to an elevation in background noise levels.
This effect arises due to the direct connection of the system to the room-
temperature electronics. In contrast to our previous work, in the study
referenced in Ref.[22], where the contribution of s-wave and p-wave
superconductivity was investigated. The critical current didn’t achieved
higher magnitudes related to p-wave superconductivity. In previous work the
sample was meticulously shielded from all forms of electromagnetic radiation
within the cavity. Additionally, it featured supplementary low-pass filters
integrated with silver paste. Nevertheless, the critical current remains large
(at s-wave level) 1(d), and the transport mode remains predominantly
ballistic, even in the presence of an additional noise source like the
antenna.

The hysteretic behavior of the $V(I)$-curves indicates Joule overheating
effects for quasiparticles [23]. This is considered a source of quasi-particle
poisoning [24]. In addition, we discuss the effect of microwave radiation in
detail. In Figs.2 (a,b,c) we show the results obtained at
$f_{\textit{RF}}=0.9$ GHz. Five representative $V(I)$-curves, corresponding to
different amplitudes of the RF radiation, are shown in Fig.2 (b); the insert
shows the $dV/dI$, which amplifies the steps of the $V(I)$-curve. The
corresponding map of the differential conductance, plotted versus the bias
current and the RF signal amplitude, is shown in Fig.2(a). The darker regions
represent voltage plateaus corresponding to a lock-in synchronization effect
between the RF field and the evolution of the superconducting phase. In other
words, the darker regions are Shapiro steps, which are numbered in Fig.2(a).

A prominent region is the large black central region which represents the
perfect superconductivity (zero voltage) of the sample. In this zero
resistance region (0-region) no steps can be observed. On the left side, this
superconducting region is bounded by the critical current dependent on the RF
amplitude $I_{C}(V_{RF})$. On the right side the 0-region is bounded by the
retrapping current which is also dependent on the RF amplitude
$I_{R}(V_{RF})$. The 0-region, if it overlaps some of the Shapiro steps, can
completely suppress them since the voltage is exactly zero in the 0-region.

For example, on the left side, and at low bias, the first visible step is
number 10, that is, the first through the ninth steps are suppressed (see
Fig.2 (b-insert)). The dependence of the critical current on the RF amplitude
and the dependence of the steps (bright regions) on the RF amplitude are both
linear, but the slope of the critical current is larger. Therefore more step
boundaries cross the critical field line at higher driving amplitudes. Thus,
at higher RF amplitudes, it is possible to observe lower-order steps. For
example, in the critical current branch, at the lowest power the lowest step
is $n=10$, while at the highest power the lowest step is $n=4$ (Fig.2(a)).

The situation with the retrapping current is more favorable for the
observation of the lower-order steps. This is simply because $I_{R}<I_{C}$, so
the voltage explores lower values on the retrapping branch of the $V(I)$-curve
(positive region in Fig.2 (a)), i.e., on the branch on which the current is
ramped down. Because of this, all steps are observed on the retrapping branch,
except the first one.

The general condition for the observation of a step number $n$ is:
$V(I_{C})<V_{n}=nhf/(2e)$ on the critical current branch and
$V(I_{R})<V_{n}=nhf/(2e)$ on the retrapping branch. On the other hand, as the
amplitude of the RF signal increases, both the critical current and the
retrapping currents are reduced. Thus the number of steps which can be
detected increases with the amplitude of the RF signal.

Another map of differential conductance is shown in Fig.2(c), which represents
$dV/dI$ plotted versus the RF amplitude and the DC voltage on the sample, $V$,
normalized by the fundamental mode voltage of the phase evolution, $hf/2e$.
Interestingly, it shows that the second step is present at very low
amplitudes. However, the first step appears only when the driving amplitude is
quite large and approaches 170 $\mu$V.

When the critical current branch is consider, up to 9 first steps appear
suppressed. This suppression is most probably due to the switching-current
voltage jump being larger than $nhf/2e$. Thus, in what follows, we focus on
the retrapping branch, where only the first step is absent, while the second
is observed.There could be multiple explanations for this phenomenon of the
first step disappearance. It could be due to the topological nature of the
proximitized material in these sample. As noted previously, Majorana fermions
support coherent transfer of single electrons and the corresponding CPR is
proportional to $\sin(\phi/2)$, where $\phi$ is the phase difference at the
junction. Such a CPR could lead to the disappearance of the first Shapiro
step. Another possibility is that the ballistic character of the electron
transport through the junction causes the second step to become more prominent
than the first one. According to the recent Galaktionov-Zaikin model [17], the
second step could occur at a lower driving power than the second step if the
phenomenon of MAR is taken into account. Yet another possible explanation is
that the second step boundary crosses the curve $I_{R}(V_{RF})$ at a lower
$V_{RF}$, where $V_{RF}$ is the amplitude of the RF driving signal. This seems
quite plausible based on the results shown on Fig.2(a).

Considering results obtained at higher frequencies, we focus on the retrapping
branch (positive current bias on all plots), since the switching current
branch produces a large jump at the critical current such that the voltage
immediately goes to value much higher than needed for the observation of low-
order steps. Yet, it is interesting to note that, at $f=2\,\mathrm{GHz}$, even
on the critical current branch, the first observed step is $n=2$, while at a
higher RF drive the $n=1$ is also visible (Fig.2(j)).

Figure 3:  Retrapping current overlapping Shapiro steps. a and b Differential
resistance versus normalized bias current and RF drive amplitude. The
retrapping currents and the switching currents (blue dashed lines) form the
main borders of a trapezoidal zero-resistance region. Microwave-induced steps
are marked by yellow lines. (a) $f=2$ GHz, The first step ($V=hf/2e$) is
observable at low RF drive. (b) $f=1$ GHz. The first step is not always
apparent. At low RF drive the junction transitions from zero resistance to the
second step ($V=hf/e$), without entering the first step.

Let us now return to the discussion regarding the retrapping branch. In this
case, we will focus on the data acquired at 1 GHz, as depicted in Fig.2(d),
(e), and (f), following the same format as previously discussed. Similarly to
the results of 0.9 GHz, the second step at this frequency appears at a lower
RF amplitude compared to the first step, as shown in Fig.2(f). This outcome is
an exact analogy to the behavior observed at 0.9 GHz.

However, the behavior of the sample changes noticeably when it moves to 1.3
GHz, as illustrated in Figs.2(g), (h), and (i). At this frequency, no steps
are observed in the retrapping current branch at the lowest $V_{RF}$ value. As
the driving amplitude increases, the first and second steps become visible
approximately at the same level of the RF drive, as depicted in Fig. 2(i).
This behavior aligns with the general expectation that an increase in the
frequency of the RF drive leads to larger step sizes ($hf/2e$) capable of
reaching the voltage level on the sample just before the retrapping event
occurs and causes the voltage to drop to zero.

Finally, the data acquired at 2 GHz is presented in Figs. 2(j), (k), and (l).
At this frequency, the multiple fractional Shapiro steps effect is not
observed. As the current increases, the first step appears at a voltage of
$V=hf/2e$, followed by another step at a higher current corresponding to
$V=hf/e$. In other words, both the first and second steps are observed. This
qualitatively distinct behavior arises from the fact that the slope of the
$I_{R}(V_{RF})$ line is now lower than the slope of the first step, whereas at
1.3 GHz, the slope of the first step line was lower than the slope of the
retrapping current line.

##  III Discussion

The studied SNS junction involves two Nb electrodes coupled through an
ultrasmall single nanocrystallite of a 3D topological insulator Bi2Te2.3Se0.7
[20], acting as the normal (N) region of the SNS junction. We present Shapiro
step maps as differential conductance $dI/dV$ plotted versus DC current bias
and RF drive power, in the frequency range from 0.9 GHz to 3 GHz. Strongly
hysteretic I(V) curves as well as the $I_{C}$(T) dependence are also shown.
The $I_{C}$(T) function is well described by the ballistic electron theory of
SNS junctions (see also ref[20]). In such measurements the Joule heating is
always present, down to the retrapping occurrence. Thus, the expectation is
that the parity of the MZM (if MZM is indeed present) should be changing very
rapidly and so the first step should be observed. Our important conclusion is
that the first step is still missing in the retrapping current branch, i.e., a
strong Joule heating, which is responsible for a pronounced hysteresis of the
V-I curves, is not able to eliminate the MFS effect. We also analyze the
critical current as a function of temperature and demonstrate that our SNS
junctions are ballistic, because of the high structural perfection of the PVD-
grown topological insulator crystals. Thus, our results provide evidence that
the incomplete even-odd effect (the MFS effect) can occur in ballistic
junctions even under strong quasiparticle poisoning. We compare the results to
the models outlined above.

A general feature, observed at all frequencies, is that all steps are visible
if the RF signal is sufficiently strong (see the top parts of the plots in
Figs.3(a,b)). The reason is that at sufficiently large $V_{RF}$ the critical
current and the retrapping current are both zero, while the superconducting
order parameter is not suppressed to zero. Under such conditions, the DC
voltage increases gradually as the DC bias current increases. Thus, all low-
order steps are visible and there are no missing steps at sufficiently large
amplitudes of the RF drive, $V_{RF}$. In Figs.3(a,b) we mark all steps with
yellow lines; the switching and retrapping currents are shown by blue dashed
lines. The main difference between the high-frequency data (2 GHz) (Fig.3(a))
and the low-frequency data (1 GHz) (Fig.3(b)) is that in the first case the
slope of the yellow lines is larger than the slope of the retrapping current
blue dashed lines, while in the latter case the slopes are about equal. Also,
the spacing between the yellow lines (the steps) is naturally lower if the RF
signal has a lower frequency, since the voltage difference between the steps
is $hf/2e$, where $f$ is the frequency of the RF signal. On the other hand, at
a high RF frequency, there is a significant region between the retrapping line
(blue) and the second yellow line. Thus, a part of the first voltage plateau
can be observed experimentally even at low RF drive (Figs.3(a,b)).

An alternative and more traditional explanation of the observed results is
given below. The hysteretic behaviour is linked to the Joule overheating
effect related to the phase winding and the corresponding voltage. This is
considered to be a source of quasiparticle poisoning. The poisoning results in
the partial observation of $4\pi$-periodic Josephson effect where only $n=\pm
1$ Shapiro steps are absent. Our results differ from those previously reported
in that we observe the absence of the first step in the retrapping current,
when the quasiparticle poisoning is not negligible.

Figure 4:  a Scheme of the two-channel thermal RSJ model. One of the two
parallel Josephson junctions (J1) stands for the trivial supercurrent and the
other one (J2) stands for the topological channel in parallel with shunting
resistance $R_{n}$. b Phase diagram of odd and even Shapiro steps in two-
channel thermal RSJ model. Simulated differential resistance map as a function
of the RF current amplitude for RF current frequencies (c) 2 GHz and (e) 0.9
GHz with 2$\pi$ and 5% 4$\pi$-component. Corresponding Shapiro bins maps for
RF current frequencies (d) 2 GHz and (f) 0.9 GHz.

We model our system using the two-channel thermal RSJ model proposed in
ref.[11, 25]. This model consists of two parts: additional coherent
superconducting channel originating from Majorana bound states and
contributing $\sin(\phi/2)$ supercurrent, and self-consistent thermal balance
which is manifested in the hysteresis of the $V(I)$ curves. In Fig.4(a) the
scheme of the two-channel thermal RSJ model is shown. There are two parallel
superconducting junctions in parallel with the shunt resistance $R$. The
Josephson junction J1 is a conventional junction with $2\pi$-CPR, and the
second junction J2 is a topological one with $4\pi$-CPR.

According to this total superconducting current is
$I_{s}(\varphi)=I^{2\pi}_{c}\sin(\varphi)+I^{4\pi}_{c}\sin(\varphi/2)$ and
equation for phase can be written as

$\frac{\hbar
d\varphi}{2eRdt}=I_{DC}+I_{RF}\sin\left(\omega_{RF}t\right)-\left(I^{2\pi}_{c}\sin(\varphi)+I^{4\pi}_{c}\sin(\varphi/2)\right),$
(1)

where IDC is the DC bias current, IRF is the RF current applied by a
generator, $\omega_{RF}$ is the frequency of RF current,
$I^{2\pi}_{c}\left(I^{4\pi}_{c}\right)$ is the critical current of $2\pi$
$(4\pi)$ component, $R$ is the normal state resistance of the junction.

To take into account the hysteretic behavior of $V(I)$ characteristics due to
Joule overheating, we write the heat balance equation:

$\langle P(t)\rangle=\Sigma U\left(T_{e}^{5}-T_{ph}^{5}\right),$ (2)

where $\Sigma$ is the electron-phonon coupling constant of the normal
material, $U$ is the effective volume of the sample, $T_{e}$ and $T_{ph}$ are
electron and phonon temperatures respectively. A self-consistent scheme is
provided as follows: at a given IDC, IRF we begin by supposing a temperature
$T$ (at the first iteration $T=T_{ph}$ is equal to the bath temperature of the
cryostat) which gives a critical current
$I_{c}(T)=I^{2\pi}_{c}(T)+I^{4\pi}_{c}(T)$. Next we solve equation 1 from
which we estimate the Joule power $P=\langle I(t)V(t)\rangle$ that gives us
temperature of electron subsystem $T_{e}=\sqrt[5]{{T_{ph}}^{5}+\frac{P}{\Sigma
U}}$ and hence a new critical current $I_{c}(T_{e})$. Then this sequence is
repeated until the difference between a new estimated temperature $T^{n+1}$
and that from the previous step $T^{n}$ is small enough. The temperature
dependence of the critical current $I_{c}(T)$ is taken from the ballistic fit
and values of parameters $\Sigma$ and $U$ are extracted from the fit of
retrapping $I_{r}$ current (see Figs.1).

As it has been shown in the work by Le Calvez et al. [11] there are two
characteristic frequencies in this system
$f^{2\pi}=\frac{1}{\hbar}2eRI^{2\pi}_{c}$ and
$f^{4\pi}=\frac{1}{\hbar}eRI^{4\pi}_{c}$. They define the 2D phase diagram,
see Fig.4(b).In Fig. 4(c,e), a numerical calculation was performed using the
model for a frequency of 0.9 GHz. The results demonstrate that at low power
levels of the RF signal, the second stage appears first, while at high power
levels, the first stage appears. To achieve the closest resemblance to the
experimental data, it was necessary to include a minimum of 5% of the
$4\pi$-periodic component. Fig.4(e,f) depicts the fitting results for a
frequency of 2 GHz, which replicate the observed behavior in the experimental
data. An interesting result of the model is that it reproduces the zero-
resistance region at low DC and RF bias and predicts a linear shape of the
boundary.

Our study sheds light on the perplexing phenomenon of selective missing of the
first Shapiro step exclusively in topological Josephson junctions. In our
ballistic Josephson junction, constructed using the 3D topological insulator
Bi2Te2.3Se0.7, this phenomenon is aptly characterized and, can be effectively
explained by a two-channel thermal RSJ model.

###### Acknowledgements.

## References

  * [1] Sauls, J.  Andreev bound states and their signatures (2018).
  * [2] Rokhinson, L. P., Liu, X. & Furdyna, J. K.  The fractional ac josephson effect in a semiconductor–superconductor nanowire as a signature of majorana particles.  _Nature Physics_ 8, 795–799 (2012).
  * [3] Mourik, V. _et al._ Signatures of majorana fermions in hybrid superconductor-semiconductor nanowire devices.  _Science_ 336, 1003–1007 (2012).
  * [4] Das, A. _et al._ Zero-bias peaks and splitting in an al–inas nanowire topological superconductor as a signature of majorana fermions.  _Nature Physics_ 8, 887–895 (2012).
  * [5] Houzet, M., Meyer, J. S., Badiane, D. M. & Glazman, L. I.  Dynamics of majorana states in a topological josephson junction.  _Physical review letters_ 111, 046401 (2013).
  * [6] Jiang, L. _et al._ Unconventional josephson signatures of majorana bound states.  _Physical review letters_ 107, 236401 (2011).
  * [7] Rosen, I. T. _et al._ Fractional ac josephson effect in a topological insulator proximitized by a self-formed superconductor.  _arXiv preprint arXiv:2110.01039_ (2021).
  * [8] Iorio, A. _et al._ Half-integer shapiro steps in highly transmissive insb nanoflag josephson junctions.  _Phys. Rev. Res._ 5, 033015 (2023).
  * [9] Wiedenmann, J. _et al._ 4 $\pi$-periodic josephson supercurrent in hgte-based topological josephson junctions.  _Nature communications_ 7, 10303 (2016).
  * [10] Bocquillon, E. _et al._ Gapless andreev bound states in the quantum spin hall insulator hgte.  _Nature Nanotechnology_ 12, 137–143 (2017).
  * [11] Le Calvez, K. _et al._ Joule overheating poisons the fractional ac josephson effect in topological josephson junctions.  _Communications Physics_ 2, 4 (2019).
  * [12] Li, C. _et al._ 4$\pi$-periodic andreev bound states in a dirac semimetal.  _Nature materials_ 17, 875–880 (2018).
  * [13] Rosenbach, D. _et al._ Reappearance of first shapiro step in narrow topological josephson junctions.  _Science advances_ 7, eabf1854 (2021).
  * [14] Domínguez, F., Hassler, F. & Platero, G.  Dynamical detection of majorana fermions in current-biased nanowires.  _Physical Review B_ 86, 140503 (2012).
  * [15] Badiane, D. M., Glazman, L. I., Houzet, M. & Meyer, J. S.  Ac josephson effect in topological josephson junctions.  _Comptes Rendus Physique_ 14, 840–856 (2013).
  * [16] Virtanen, P. & Recher, P.  Microwave spectroscopy of josephson junctions in topological superconductors.  _Physical Review B_ 88, 144507 (2013).
  * [17] Galaktionov, A. V. & Zaikin, A. D.  Fractional shapiro steps without fractional josephson effect.  _Physical Review B_ 104, 054521 (2021).
  * [18] Bae, M.-H., Dinsmore III, R. C., Aref, T., Brenner, M. & Bezryadin, A.  Current-phase relationship, thermal and quantum phase slips in superconducting nanowires made on a scaffold created using adhesive tape.  _Nano letters_ 9, 1889–1896 (2009).
  * [19] Beenakker, C.  Annihilation of colliding bogoliubov quasiparticles reveals their majorana nature.  _Physical review letters_ 112, 070604 (2014).
  * [20] Stolyarov, V. S. _et al._ Josephson current mediated by ballistic topological states in bi2te2. 3se0. 7 single nanocrystals.  _Communications Materials_ 1, 38 (2020).
  * [21] Yakovlev, D. S. _et al._ Physical vapor deposition features of ultrathin nanocrystals of bi2 (te x se1–x) 3.  _The Journal of Physical Chemistry Letters_ 13, 9221–9231 (2022).
  * [22] Stolyarov, V. S. _et al._ Resonant oscillations of josephson current in nb-bi2te2. 3se0. 7-nb junctions.  _Advanced Quantum Technologies_ 5, 2100124 (2022).
  * [23] Courtois, H., Meschke, M., Peltonen, J. & Pekola, J. P.  Origin of hysteresis in a proximity josephson junction.  _Physical review letters_ 101, 067002 (2008).
  * [24] Karzig, T., Cole, W. S. & Pikulin, D. I.  Quasiparticle poisoning of majorana qubits.  _Physical Review Letters_ 126, 057702 (2021).
  * [25] De Cecco, A., Le Calvez, K., Sacépé, B., Winkelmann, C. & Courtois, H.  Interplay between electron overheating and ac josephson effect.  _Physical Review B_ 93, 180505 (2016).