> under the concept; though for us men, with our defective knowledge, the
> question may not always be decidable. We may express this metaphorically as
> follows: the concept must have a sharp boundary.’ (Frege, 1903, §56)

This is vintage Frege (and surely also early Wittgenstein): definitions should
be unambiguous and the world is a place in which every _Gegenstand_ (= object)
is sharply defined by its properties.103103103Continuing footnote 101: Used in
set theory, unlimited definition by extension eventually led to Frege’s
downfall, since his comprehension principle leads to Russell’s paradox. This
is also inherent in the logical atomism of Russell (and the _Tractatus_ ,
whose world is modeled on propositional logic and its binary truth tables).
Against this (and the entire preceding tradition in philosophy), the later
Wittgenstein forwarded his famous concept of a _family resemblance_
:104104104As noted for example by Sluga (2006), to understand Wittgenstein’s
motivation for the idea of a family resemblance it is very useful to go back
to an earlier passage from the _Blue Book_ (pp. 19–20): ‘The idea that in
order to get clear about the meaning of a general term one had to find the
common element in all its applications has shackled philosophical
investigation; for it has not only led to no result, but also made the
philosopher dismiss as irrelevant the concrete cases, which alone could have
helped him to understand the usage of the general term. When Socrates asks the
question, “what is knowledge?” he does not even regard it as a preliminary
answer to enumerate cases of knowledge.’ Essay XI in Baker & Hacker (2009a)
provides a historical perspective on family resemblances. Nietzsche already
described a ‘family resemblance’ between specific groups of languages and even
ways of philosophizing in these languages. But: ‘It is one thing to family
resemblances between different languages and to group the various languages
into families according to their genesis. It is quite a different thing,
however, to extend the notion of a family resemblance to _concepts_ (including
the concept of language), i.e. to argue that the extension of a concept may be
united not by common characteristics but by overlapping similarities between
the members.’ (Baker & Hacker, 2009a, p. 210).

> Hier stoßen wir auf die große Frage, die hinter allen diesen Betrachtungen
> steht. – Denn man könnte mir nun einwenden: “Du machst dir’s leicht! Du
> redest von allen möglichen Sprachspielen, hast aber nirgends gesagt, was
> denn das Wesentliche des Sprachspiels, und d.h. der Sprache, ist. Was allen
> diesen Vorgängen gemeinsam ist und sie zur Sprache, oder zu Teilen der
> Sprache macht. Du schenkst dir also gerade den Teil der Untersuchung, der
> dir selbst seinerzeit das meiste Kopfzerbrechen gemacht hat, nämlich den,
> die allgemeine Form des Satzes und der Sprache betreffend.” Und das ist
> wahr. – Statt etwas anzugeben, was allem, was wir Sprache nennen, gemeinsam
> ist, sage ich, es ist diesen Erscheinungen gar nicht Eines gemeinsam,
> weswegen wir für alle das gleiche Wort verwenden, – sondern sie sind
> miteinander in vielen verschiedenen Weisen verwandt. Und dieser
> Verwandtschaft, oder diesen dieser Verwandtschaften wegen nennen wir sie
> alle “Sprachen”. Ich will versuchen, dies zu erklären. (…)
>
> Wir sehen ein kompliziertes Netz von Ähnlichkeiten, die einander übergreifen
> und kreuzen. Ähnlichkeiten im Großen und Kleinen. (…) Ich kann diese
> Ähnlichkeiten nicht besser charakterisieren, als durch das Wort
> “Familienähnlichkeiten”; denn so übergreifen und kreuzen sich die
> verschiedenen Ähnlichkeiten, die zwischen den Gliedern einer Familie
> bestehen: Wuchs, Gesichtszüge, Augenfarbe, Gang, Temperament, etc. etc. –
> Und ich werde sagen: die ‘Spiele’ bilden eine Familie.105105105‘Here we come
> up against the great question that lies behind all these considerations.–
> For someone might object against me: “You take the easy way out! You talk
> about all sorts of language-games, but have nowhere said what the essence of
> a language-game, and hence of language, is: what is common to all these
> activities, and what makes them into language or parts of language. So you
> let yourself off the very part of the investigation that once gave you
> yourself most headache, the part about the general form of propositions and
> of language.” And this is true.–Instead of producing something common to all
> that we call language, I am saying that these phenomena have no one thing in
> common which makes us use the same word for all, – but that they are related
> to one another in many different ways. And it is because of this
> relationship, or these relationships, that we call them all “language”. I
> will try to explain this. (§65) we see a complicated network of similarities
> overlapping and criss-crossing: sometimes overall similarities, sometimes
> similarities of detail. (§66) I can think of no better expression to
> characterize these similarities than “family resemblances”; for the various
> resemblances between members of a family: build, features, colour of eyes,
> gait, temperament, etc. etc. overlap and criss-cross in the same way.– And I
> shall say: ‘games’ form a family.’ (§67)  (Wittgenstein, 1984a, §§65–67).

Except for some rather vague allusions involving numbers and propositions
(Baker & Hacker, 2009a, §XI.7) Wittgenstein did not apply this idea to
mathematics. There are two relevant layers:

  1. 1.

Like a (language) game, mathematics _as a whole_ could be seen as a family
resemblance.

  2. 2.

Concepts _within_ mathematics, notably definitions and proofs, might be family
resemblances.

The first would mean that mathematics is a concept with various different
instantiations, such as Babylonian mathematics, Chinese mathematics, Greek
mathematics, 17th century European mathematics, 18th (…), 19th (…), and 20th
century mathematics (finally a homogeneous worldwide practice) are seen as
different members of the mathematical family. This is a possible view, but in
the context of our question to what extent mathematics may be seen as a game
(which is the driving force of this article), we prefer to see mathematics as
a single notion, which we interpet as a “motley of language games”. _That_ is
a family resemblance, which, switching to the second point, in case of
mathematics for example involves axioms (§A.2), definitions, and proofs
(§A.4).

All of these, but especially the second, are family resemblances. Indeed, one
finds all kinds of definitions in mathematics. See Coumans (2023), p. 212, for
a classification of definitions.

###  A.4 Proofs

In the formal(ist) language game played by Hilbert, proofs are examples or
rule following. But where do the rules (of deduction) come from? Hilbert
answers this in a famous passage:106106106Many other authors have used this,
or parts of it; e.g. Detlefsen (1986), p. x; Mühlhölzer, 2010, p. 74.

> Das Formelspiel, über das Brouwer so wegwerfend urteilt, hat außer dem
> mathematischen Wert noch eine wichtige allgemeine philosophische Bedeutung.
> Dieses Formelspiel vollzieht sich nämlich nach gewissen bestimmten Regeln,
> in denen die _Technik unseres Denkens_ zum Ausdruck kommt. Diese Regeln
> bilden ein abgeschlossenes System, das sich auffinden und endgültig angeben
> läßlt. Die Grundidee meiner Beweistheorie ist nichts anderes, als die
> Tätigkeit unseres Verstandes zu beschreiben, ein Protokoll über die Regeln
> aufzunehmen, nach denen unser Denken tatsächlich verfährt. Das Denken
> geschieht eben parallel dem Sprechen und Schreiben, durch Bildung und
> Aneinanderreihung von Sätzen. Wenn irgendwo eine Gesamtheit von
> Beobachtungen und Erscheinungen verdient, zum Gegenstand einer ernsten und
> gründlichen Forschung gemacht zu werden, so ist es diese hier — liegt es
> doch in der Aufgabe der Wissenschaft, uns von Willkür, Gefühl und Gewöhnung
> freizumachen und vor dem Subjektivismus zu bewahren, der sich schon in den
> Anschauungen Kroneckers bemerkbar gemacht hat und der, wie mir scheint, in
> dem Intuitionismus seinen Gipfelpunkt erreicht.107107107 ‘The formula game
> that Brouwer so deprecates has, besides its mathematical value, an important
> general philosophical significance. For this formula game is carried out
> according to certain definite rules, in which the technique of our thinking
> is expressed. These rules form a closed system that can be discovered and
> definitively stated. The fundamental idea of my proof theory is none other
> than to describe the activity of our understanding, to make a protocol of
> the rules according to which our thinking actually proceeds. Thinking, it so
> happens, parallels speaking and writing: we form statements and place them
> one behind another. If any totality of observations and phenomena deserves
> to be made the object of a serious and thorough investigation, it is this
> one-since, after all, it is part of the task of science to liberate us from
> arbitrariness, sentiment, and habit and to protect us from the subjectivism
> that already made itself felt in Kronecker’s views and, it seems to me,
> finds its culmination in intuitionism.’ (Hilbert, 1928, pp. 79–80)

Hilbert’s view on the origin of logic (which is his and Frege’s tool for
deductive proofs in mathematics), then, is that its rules express the
_technique of our thinking_. This view keeps Hilbert at some distance from
both Frege and Wittgenstein. Frege emphatically insisted that the laws of
logic are _not_ the psychological laws by which we think and reason, but the
laws by which reasoning is justified (Soames, 2014, p. 30), or even the most
general laws of truth, which as such are _normative_ for correct thinking
(rather than _descriptive_ of it), for which ordinary language is too
imprecise (Kuusela, 2019a, §I.1). For Wittgenstein, on the other hand, the
‘technique of thinking’ is grounded in practice and it is this practice that
leads to the laws of logic (Mühlhölzer, 2010, p. 75). Claiming that practice
is ruled by our way of thinking, Hilbert seems to put Wittgenstein’s view on
its head, as the latter maintained that our way of thinking is ruled by some
practice.

This seems a long way from Frege’s lofty faith in the supreme status of logic,
which in turn encouraged Russell (who followed Frege in many ways, but left
him at the bitter end) to write:

> The old logic put thought in fetters, while the new logic gives it wings. It
> has, in my opinion, introduced the same kind of advance into philosophy as
> Galileo introduced into physics, making it possible at last to see what
> kinds of problems may be capable of solution, and what kinds must be
> abandoned as beyond human powers. (…) It is in this way that the study of
> logic becomes the central study in philosophy: it gives the method of
> research in philosophy, just as mathematics gives the method in physics. And
> as physics, which, from Plato to the Renaissance, was as unprogressive, dim,
> and superstitious as philosophy, became a science through Galileo’s fresh
> observation of facts and subsequent mathematical manipulation, so
> philosophy, in our own day, is becoming scientific through the simultaneous
> acquisition of new facts and logical methods. (Russell, 1914, pp. 48, 194)

See also Kuusela (2019a), p. 14. But if, following PI §§130–131, we regard
language games as logical tools of examination or comparison in the study of
language (and hence in philosophy); and, as advocated in this paper, regard
mathematics as consisting of specific language games that can be used in the
study of physics in a very similar way, namely as _yardsticks_ rather than
_descriptions_ , then this late Wittgensteinian perspective seems to confirm
Russell’s analogy between the use of logic as _the_ method in philosophy and
the use of mathematics as _the_ method in physics.

_Acknowledgement:_ This work originated in a B.Sc. thesis of the second author
(Singh, 2013), supervised by the first, which was greatly expanded afterwards.
The authors are grateful to Jeremy Butterfield, Thomas Friederich, Fred
Muller, Tomasz Placek, and Freek Wiedijk for comments and further help with
this paper.

## References

  * [1] Abelson, R. (1967). Definition. _Encyclopedia of Philosophy Second Edition, Volume 2_ , ed. Edwards, P., pp. 664–677 (Macmillan).
  * [2] Appel, K., Haken, W. (1977). The solution of the four-color-map problem. _Scientific American_ 237, 108–121. https://www.jstor.org/stable/24953967.
  * [3] Avigad, J. (2021). Reliability of mathematical inference. _Synthese_ 198, 7377–7399. https://doi.org/10.1007/s11229-019-02524-y.
  * [4] Baker, G.P., Hacker, P.M.S. (2009a). _Wittgenstein: Understanding and Meaning: Volume 1 of an Analytical Commentary on the Philosophical Investigations. part i: essays_. Second Edition (Wiley–Blackwell).
  * [5] Baker, G.P., Hacker, P.M.S. (2009b). _Wittgenstein: Understanding and Meaning: Volume 1 of an Analytical Commentary on the Philosophical Investigations. part ii: exegesis §§1–184._ Second Edition (Wiley–Blackwell).
  * [6] Baker, G.P., Hacker, P.M.S. (2009c). _Wittgenstein: Rules, Grammar and Necessity: Volume 2 of an Analytical Commentary on the Philosophical Investigations. essays and exegesis of §§185–242_. Second Edition (Wiley–Blackwell).
  * [7] Balaguer, M. (1998). _Platonism and Anti-platonism in Mathematics_ (Oxford University Press).
  * [8] Baldwin, J.T. (2018). _Model Theory and the Philosophy of Mathematical Practice: Formalization Without Foundationalism_ (Cambridge University Press).
  * [9] Bangu, S. (undated). Ludwig Wittgenstein: Later philosophy of mathematics. _Internet Encyclopedia of Philosophy_. https://iep.utm.edu/wittmath/.
  * [10] Barco, A. (2018). _On Mathematical Truth: Wittgenstein and the Dissolution of Benacerraf’s Dilemma_. PhD Thesis, University of Southampton. https://eprints.soton.ac.uk/424356/.
  * [11] Benacerraf, P. (1973). Mathematical truth. _The Journal of Philosophy_ 70, 661–679. https://doi.org/10.2307/2025075.
  * [12] Blanchette, P. (2017). Models in geometry and logic: 1870–1920. _Logic, Methodology and Philosophy of Science: Proceedings of the 15th International Congress_ , ed. Sober, N.S., pp. 41–61. (College Publications). https://philarchive.org/archive/BLAMIG.
  * [13] Blanchette, P. (2018). The Frege–Hilbert controversy. _Stanford Encyclopedia of Philosophy (Fall 2018 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/fall2018/entries/frege-hilbert/.
  * [14] Blumenthal, O. (1970). Lebensgeschichte. _David Hilbert: Gesammelte Abhandlungen, Band III, 2. Auflage_ , pp. 388–429 (Springer).
  * [15] Boole, G. (1847). _The Mathematical Analysis of Logic: Being an Essay Towards a Calculus of Deductive Reasoning_ (MacMillan, Barclay, & MacMillan). https://www.gutenberg.org/ebooks/36884.
  * [16] Bostock, D. (2009). _Philosophy of Mathematics: An Introduction_ (Wiley–Blackwell).
  * [17] Bühler, W.K. (1981). _Gauss: A Biographical Study_ (Springer).
  * [18] Burgess, J., De Toffoli, S. (2022). What is mathematical rigor? https://philarchive.org/archive/BURWIM-6.
  * [19] Burke, M. (2015). _Frege, Hilbert, and Structuralism_. PhD Thesis, University of Ottawa. https://ruor.uottawa.ca/bitstream/10393/31937/1/Burke_Mark_2015_thesis.pdf.
  * [20] Button, T., Walsh, S. (2018). _Philosophy and Model Theory_ (Oxford University Press).
  * [21] Cellucci, C. (2018). Definition in mathematics. _European Journal for the Philosophy of Science_ 8, 605–629. https://doi.org/10.1007/s13194-018-0203-y.
  * [22] Connes, A., with Lichnerowicz, A., Schützenberger, A.P. (2001). _Triangle of Thoughts_ (AMS).
  * [23] Corry, L. (2004). _David Hilbert and the Axiomatization of Physics (1898–1918): From Grundlagen der Geometrie to Grundlagen der Physik_ (Kluwer).
  * [24] Corry, L. (2018). Hilbert’s sixth problem: Between the foundations of geometry and the axiomatization of physics. _Philosophical Transactions of the Royal Society_ A376:20170221. https://doi.org/10.1098/rsta.2017.0221.
  * [25] Coumans, V. (2023). _Studies on Definitions in Mathematical Practice_. PhD Thesis, Radboud University. https://repository.ubn.ru.nl/handle/2066/291649.
  * [26] Dafermos, M., Holzegel, G., Rodnianski, I. (2019). The linear stability of the Schwarzschild solution to gravitational perturbations. _Acta Mathematica_ 222, 1–214. https://doi.org/10.4310/ACTA.2019.v222.n1.a1.
  * [27] Dafermos, M., Holzegel, G., Rodnianski, Taylor, M. (2021). The non-linear stability of the Schwarzschild family of black holes. arXiv:2104.08222.
  * [28] David, M. (2022). The correspondence theory of truth. _The Stanford Encyclopedia of Philosophy (Summer 2022 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/sum2022/entries/truth-correspondence/.
  * [29] Dawson, R. (2014). ] Wittgenstein on pure and applied mathematics. _Synthese_ 191, 4131–4148. https://doi.org/10.1007/s11229-014-0520-4.
  * [30] Dehnel, P. (2020). The middle Wittgenstein’s critique of Frege. _International Journal of Philosophical Studies_ 28, 75–95. https://doi.org/10.1080/09672559.2020.1716384.
  * [31] Detlefsen, M. (1986). _Hilbert’s Program_ (Reidel).
  * [32] Detlefsen, M. (2005). Formalism. _The Oxford Handbook of Philosophy of Mathematics and Logic_ , ed. Shapiro, S., pp. 236–317 (Oxford University Press).
  * [33] Diamond, C., ed. (1975). _Wittgenstein’s Lectures on the Foundations of Mathematics, Cambridge 1939_ (University of Chicago Press).
  * [34] Dieudonné, J. (1971). Modern axiomatic method and the foundations of mathematics. _Great Currents of Mathematical Thought, Volume 2_ , ed. le Lionnais, F., pp. 251–266 (Dover/Constable).
  * [35] Dominus, M. (2006). The Universe of Discourse. https://blog.plover.com/math/PM.html.
  * [36] Dutilh Novaes, C. (2011). The different ways in which logic is (said to be) formal. _History and Philosophy of Logic_ 32, 303–332. https://doi.org/10.1080/01445340.2011.555505.
  * [37] Epple, M. (1994). Das bunte Geflecht der mathematischen Spiele: Ein Diskurs über die Natur der Mathematik. _Mathematische Semesterberichte_ 41, 113–133. https://doi.org/10.1007/BF03186505.
  * [38] Ewald, W., Hallett, M., Majer, U., Sieg, W. (2012). _David Hilbert’s Lectures on the Foundations of Arithmetic and Logic 1894-1917_ (Springer).
  * [39] Ewald, W., Sieg. W., eds. (2013). _David Hilbert’s Lectures on the Foundations of Arithmetic and Logic 1917–1933_ (Springer).
  * [40] Feferman, S. (1999). Does mathematics need new axioms? _American Mathematical Monthly_ 106, 99–111. https://www.jstor.org/stable/420965.
  * [41] Feferman, S., Friedman, H.M., Maddy, P., Steel, J.R. (2000). Does mathematics need new axioms? _Bulletin of Symbolic Logic_ 6, 401–446. https://www.jstor.org/stable/420965.
  * [42] Ferreirós, J. (2008). _Labyrinth of Thought: A History of Set Theory and its Role in Modern Mathematics_ (Springer).
  * [43] Ferreirós, J. (2016). _Mathematical Knowledge and the Interplay of Practices_ (Princeton University Press).
  * [44] Floyd, J. (2015). Critical Studies/Book Reviews: Depth and Clarity (Review of Felix Mühlhölzer, _Braucht die Mathematik eine Grundlegung?_. _Philosophia Mathematica (III)_ 23, 255–294. https://doi.org/10.1093/philmat/nku037.
  * [45] Floyd, J. (2021). _Wittgenstein’s Philosophy of Mathematics_ (Cambridge University Press).
  * [46] Floyd, J. (2023). “Surveyability” in Hilbert, Wittgenstein and Turing. _Philosophies_ 8:6. https://doi.org/10.3390/philosophies8010006.
  * [47] Franks, C. (2009). _The Autonomy of Mathematical Knowledge: Hilbert’s Program Revisited_ (University of Notre Dame Press).
  * [48] Franzén, T. (2005). _Gödel’s Theorem: An Incomplete Guide to Its Use and Abuse_ (CRC Press).
  * [49] Frege, G. (1879). _Begriffsschrift: Eine der arithmetischen nachgebildete Formelsprache des reinen Denkens_ (Louis Nebert). Translation: _Conceptual Notation and Related Articles_ , ed. Bynam, T.W. (Oxford University Press, 1972).
  * [50] Frege, G. (1884). _Die Grundlagen der Arithmetik: Eine logisch-mathematische Untersuchung über den Begriff der Zahl_ (W. Koebner). Translation: _The Foundations of Mathematics: A Logico-Mathematical Enquiry into the Concept of Number, Revised 2nd Edition_ (Blackwell, 1974).
  * [51] Frege, G. (1892). Über Sinn und Bedeutung. _Zeitschrift für Philosophie und philosophische Kritik_ 100, 25–50. https://www.deutschestextarchiv.de/book/show/frege_sinn_1892.
  * [52] Frege, G. (1899). Über die Zahlen des Herrn H. Schubert. Reprinted in: Frege, G. (1966). _Logische Untersuchungen: Einleitung von Günther Patzig_ , pp. 113–138 (Vandenhoeck & Ruprecht). Frege, G. Kleine Schriften.
  * [53] Frege, G. (1903). _Grundgesetze der Arithmetik, Band II_ (Pohle). Translation: _Basic Laws of Arithmetic_ , eds. Ebert, P.A., Rossberg, M., Wright, C. (Oxford University Press, 2013).
  * [54] Frege, G (1906). Antwort auf die Ferienplauderei des Herrn Thomae. _Jahresbericht der Deutschen Mathematiker-Vereinigung_ 15, 586–592. http://resolver.sub.uni-goettingen.de/purl?PPN37721857X_0015.
  * [55] Frege, G (1908a). Die Unmöglichkeit der Thomaeschen formalen Arithmetik aufs neue nachgewiesen. _Jahresbericht der Deutschen Mathematiker-Vereinigung_ 17, 52–55. http://resolver.sub.uni-goettingen.de/purl?PPN37721857X_0017.
  * [56] Frege, G (1908b). Schlußbemerkung. _Jahresbericht der Deutschen Mathematiker-Vereinigung_ 17, 56. http://resolver.sub.uni-goettingen.de/purl?PPN37721857X_0017.
  * [57] Friederich, S. (2011). Motivating Wittgenstein’s perspective on mathematical sentences as norms. _Philosophia Mathematica_ 19, 1–19. https://doi.org/10.1093/philmat/nkq024.
  * [58] Friederich, S. (2014). Warum die Mathematik keine ontologische Grundlegung braucht: Wittgenstein und die axiomatische Methode. _Wittgenstein-Studien_ 5, 163–178. https://doi.org/10.1515/wgst.2014.5.1.163.
  * [59] Gabriel, G., Kambartel, F., Thiel, C., eds. (1980). _Gottlob Freges Briefwechsel_ (Felix Meiner Verlag). Translation: _Gottlob Frege: Philosophical and Mathematical Correspondence_ , eds. Gabriel, G., Hermes, H., Kambartel, F., Thiel, C., Veraart, A., McGuinness, B., Kaal, H. (Blackwell, 1980).
  * [60] Geach, P.T., Black, M., eds. (1960). _Translations from the Philosophical Writings of Gottlob Frege. Second Edition_ (Blackwell).
  * [61] Gerrard, S. (1991). Wittgenstein’s philosophies of mathematics. _Synthese_ 87, 125–142. https://doi.org/10.1007/BF00485331.
  * [62] Gerrard, S. (1996). A philosophy of mathematics between two camps. _The Cambridge Companion to Wittgenstein_ , eds. Sluga, H., Stern, D.G., pp. 171–197 (Cambridge University Press).
  * [63] Geuvers, H. (2009). Proof assistants: History, ideas and future. _Sadhana_ 34, 3–25. https://doi.org/10.1007/s12046-009-0001-5.
  * [64] Giovannini, E.N., Schiemer, G. (2021). What are implicit definitions? _Erkenntnis_ 86, 1661–1691. https://doi.org/10.1007/s10670-019-00176-5.
  * [65] Gonthier, G. (2008). Formal proof — The four-color theorem. _Notices of the AMS_ 55, 1382–1393. https://www.ams.org/notices/200811/tx081101382p.pdf?referer=www.clickfind.com.au.
  * [66] Gowers, T. (2008). _The Princeton Companion of Mathematics_ (Princeton University Press).
  * [67] Gray, L. (2008). _Plato’s Ghost: The Modernist Transformation of Mathematics_ (Princeton University Press.
  * [68] Gupta, A., Mackereth, S. (2023). Definitions. _The Stanford Encyclopedia of Philosophy (Fall 2023 Edition)_ , eds. Zalta, E.N., Nodelman, U. https://plato.stanford.edu/archives/fall2023/entries/definitions/.
  * [69] Haken, W. (2006). An attempt to understand the four-color theorem. _Journal of Graph Theory_ 1, 193–206. https://doi.org/10.1002/jgt.3190010304.
  * [70] Hallett, M. (2010). Frege and Hilbert. _Cambridge Companion to Frege_ , eds. Potter, M., Ricketts, T., pp. 413–464 (Cambridge University Press).
  * [71] Hallett, M. Majer, U. (2004). _David Hilbert’s Lectures on the Foundations of Geometry 1891–1902_ (Springer).
  * [72] Hamami, Y. (2018). Mathematical inference and logical inference. _The Review of Symbolic Logic_ 11, 665–704. https://doi.org/10.1017/S1755020317000326.
  * [73] Hamami, Y., Morris, R.L. (2020). Philosophy of mathematical practice: a primer for mathematics educators. _ZDM_ 52, 1113–1126. https://doi.org/10.1007/s11858-020-01159-5.
  * [74] Hamami, Y., Morris, R.L. (2023). Rationality in mathematical proofs. _Australasian Journal of Philosophy_. https://doi.org/10.1080/00048402.2022.2118338.
  * [75] Hardy, G.H. (1929). Mathematical Proof. _Mind, New Series_ 38, 1–25. https://www.jstor.org/stable/2249221.
  * [76] Hardy, G.H. (1940). _A Mathematician’s Apology_ (Cambridge University Press). https://archive.org/details/hardy_annotated.
  * [77] Hawkins, N., Potter, M. (2022). _Wittgenstein: Interview with Prof. Michael Potter_. https://www.youtube.com/watch?v=-4O4hUwcpDw.
  * [78] Heine, E. (1872). Die Elemente der Functionenlehre. 1. _Journal für die Reine und Angewandte Mathematik_ 74, 172–188. https://doi/org/10.1515/crll.1872.74.172.
  * [79] Heintz, B. (2000). _Die Innenwelt der Mathematik: Zur Kultur und Praxis einer beweisenden Disziplin_ (Springer).
  * [80] Hempel, C.G. (1952). Fundamentals of concept formation in empirical science. _International Encyclopedia of Unified Science_ , ed. Neurath, O., Vol. II. no. 7, pp. 1–93 (Chicago University Press).
  * [81] Hilbert, D. (1890). Über die Theorie der algebraischen Formen. _Mathematische Annalen_ 36, 473–534. https://doi.org/10.1007/BF01208503.
  * [82] Hilbert, D. (1893). Üeber die vollen Invariantensysteme. _Mathematische Annalen_ 42, 313–373. https://doi.org/10.1007/BF01444162.
  * [83] Hilbert, D. (1899). _Grundlagen der Geometrie_ (Teubner). Translation: _Foundations of Geometry_ (Open Court, 1950). https://math.berkeley.edu/~wodzicki/160/Hilbert.pdf
  * [84] Hilbert, D. (1900). Mathematische Probleme. _Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-Physikalische Klasse_ 1900, 253–297. http://eudml.org/doc/58479. Translation: Mathematical problems, _Bulletin of the American Mathematical Society_ 8, 437–479 (1902). Reprinted in _Bulletin of the American Mathematical Society (New Series)_ 37, 407–436 (2000).
  * [85] Hilbert, D. (1918). Axiomatisches Denken. _Mathematische Annalen_ 78, 405–415. Translation: Axiomatic thought, _From Kant to Hilbert: A Source Book in the Foundations of Mathematics, Vol. II_ , ed. Ewald, W., pp. 1105–1115 (Clarendon Press, 1996).
  * [86] Hilbert, D. (1922a). Die logischen Grundlagen der Mathematik. _Mathematische Annalen_ 88, 151–165.
  * [87] Hilbert, D. (1922b). Neubegründung der Mathematik. Erste Mitteilung. _Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg_ 1, 157–177. https://doi.org/10.1007/BF02940589. Translation: The new grounding of mathematics, _From Kant to Hilbert: A Source Book in the Foundations of Mathematics, Vol. II_ , ed. Ewald, W., pp. 1115–1133 (Clarendon Press, 1996).
  * [88] Hilbert, D. (1926). Über das Unendliche. _Mathematische Annalen_ 95, 161–190. https://link.springer.com/content/pdf/10.1007/BF01206605.pdf. Translation: On the infinite, _Philosophy of Mathematics: Selected Readings, Second Edition_ , eds. Benacerraf, P., Putnam, H., pp. 183–201 (Cambridge University Press, 1983).
  * [89] Hilbert, D. (1928). Die grundlagen der mathematik: Vortrag, gehalten auf Einladung des Mathematischen Seminars im Juli 1927 in Hamburg. _Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg_ 6, 65–85. https://doi.org/10.1007/BF02940602. Translation: The foundations of mathematics. _From Frege to Gödel: A Source Book in Mathematical Logic, 1879–1931_ , ed. van Heijenoort, J., pp. 464–479 (Harvard University Press, 1967). https://www.marxists.org/reference/subject/philosophy/works/ge/hilbert.htm.
  * [90] Hilbert, D. (1992). _Natur und mathematisches Erkennen: Vorlesungen, gehalten 1919–1920 in Göttingen, nach der Ausarbeitung von Paul Bernays herausgegeben von David E. Rowe_ (Springer).
  * [91] Hilbert, D., Ackermann, W. (1928). _Grundzüge der theoretischen Logik_ (Springer).
  * [92] Hilbert, D., Bernays, P. (1934, 1939). _Grundlagen der Mathematik. I, II._ (Springer).
  * [93] Hodges, W. (1993). _Model Theory_ (Cambridge University Press).
  * [94] Hohwy, J. (2013). _The Predictive Mind_ (Oxford University Press).
  * [95] Illigens, E. ( 1893). _Die unendliche Anzahl und die Mathematik_ (Theissing).
  * [96] Katz, E. (2018). The mixed mathematical intermediates. _Plato Journal_ 18, 83–96. https://doi.org/10.14195/2183-4105_18_7.
  * [97] Kennedy, J. (2020). Kurt Gödel. _The Stanford Encyclopedia of Philosophy (Winter 2020 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/win2020/entries/goedel/.
  * [98] Kienzler, W. (1997). _Wittgensteins Wende zu zeiner Spätphilosophie 1930–1932: Eine historische und systematische Darstellung_ (Suhrkamp).
  * [99] Kienzler, W. (2012). Wittgenstein and Frege. _The Oxford Handbook on Wittgenstein_ , eds. Kuusela, O., McGinn, M., Chapter 4 (Oxford University Press).
  * [100] Kienzler, W. (2013). Wittgenstein reads Plato. _Wittgenstein and Plato: Connections, Comparisons and Contrasts_ , eds. Presissinotto, L., Ramón Cámara, B., pp. 25–47 (Palgrave Macmillan).
  * [101] Kragh, H. (2015). Mathematics and physics: The idea of a pre-established harmony. _Science and Education_ 24, 515–527. https://doi.org/10.1007/s11191-014-9724-8.
  * [102] Krömer, R. (2007). _Tool and Object: A History and Philosophy of Category Theory_ (Birkhäuser).
  * [103] Künne, W. (2003). _Conceptions of Truth_ (Clarendon Press).
  * [104] Kuusela, O. (2019a). _Wittgenstein on Logic as the Method of Philosophy_ (Oxford University Press).
  * [105] Kuusela, O. (2019b). Wittgenstein’s reception of Socrates. _Brill’s Companion to the Reception of Socrates_ , ed. Moore, C., pp. 883–907 (Brill).
  * [106] Landry, E. (2023). _Plato Was Not a Mathematical Platonist_ (Cambridge University Press).
  * [107] Lawrence, R. (2023). Frege, Thomae, and Formalism: Shifting perspectives. _Journal for the History of Analytical Philosophy_ 11.2. https://doi.org/10.15173/jhap.v11i2.5366.
  * [108] Linnebo, Ø. (2017). _Philosophy of Mathematics_ (Princeton University Press).
  * [109] Linnebo, Ø. (2023). Platonism in the philosophy of mathematics. _The Stanford Encyclopedia of Philosophy (Summer 2023 Edition)_ , eds. Zalta, E.N., Nodelman, U. https://plato.stanford.edu/archives/sum2023/entries/platonism-mathematics/.
  * [110] Mancosu, P. (2008). _The Philosophy of Mathematical Practice_ (Oxford University Press).
  * [111] Maddy, P. (2008). How applied mathematics became pure. _The Review of Symbolic Logic_ 1, 16–41. https://doi.org/10.1017/S1755020308080027.
  * [112] Majer, U. (2001). The axiomatic method and the foundations of science: Historical roots of mathematical physics in Göttingen (1900–1930). _John von Neumann and the Foundations of Quantum Physics_ , eds. Rédei, M., Stöltzner, M., pp 11–33 (Springer). https://doi.org/10.1007/978-94-017-2012-0_3.
  * [113] Majer, U. (2006). Hilbert’s axiomatic approach to the foundations of science—A failed research program? _Boston Studies in the Philosophy of Science_ 251, 155–183. https://link.springer.com/content/pdf/10.1007/978-1-4020-5195-1_5.pdf.
  * [114] Majer, U. (2014). The “axiomatic method” and its constitutive role in physics. _Perspectives on Science_ 22, 56–79. https://doi.org/10.1162/POSC_a_00118.
  * [115] Majer, U., Sauer, T. (2006). Intuition and the axiomatic method in Hilbert’s foundation of physics: Hilbert’s idea of a recursive epistemology in his third Hamburg lecture. _Intuition and the Axiomatic Method_ , eds. Carson, E., Huber, R., pp. 213–233 (Springer).
  * [116] Marion, M. (1998). _Wittgenstein, Finitism, and the Foundations of Mathematics_ (Clarendon Press).
  * [117] Marquis, J.P. (2009). _From a Geometrical Point of View: A Study of the History and Philosophy of Category Theory_ (Springer).
  * [118] Max, I. (2020). “Denken wir wieder an die Intention, Schach zu spielen.” Zur Rolle von Schachanalogien in Wittgensteins Philosophie ab 1929. _Wittgenstein-Studien Band 11, Spezialsektion: Wittgenstein über das Psychische_ , ed. Raatzsch, R., pp. 183–206 (De Gruyter). https://doi.org/10.1515/witt-2020-0010.
  * [119] Mehrtens, H. (1990). _Moderne Sprache Mathematik: Eine Geschichte des Streits um die Grundlagen der Disziplin und des Subjekts formaler Systeme_ (Suhrkamp).
  * [120] Mendell, H. (2019). Aristotle and mathematics. _The Stanford Encyclopedia of Philosophy (Fall 2019 Edition)_ , ed. Zalta, E.N. (ed.). https://plato.stanford.edu/archives/fall2019/entries/aristotle-mathematics/.
  * [121] Millidge, B., Seth, A., Buckley, C.L. (2022). Predictive coding: a theoretical and experimental review. https://arxiv.org/pdf/2107.12979.
  * [122] Monton, B., Mohler, C. (2021). Constructive empiricism. _The Stanford Encyclopedia of Philosophy (Summer 2021 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/sum2021/entries/constructive-empiricism/.
  * [123] Mühlhölzer, F. (2006). “A mathematical proof must be surveyable”: What Wittgenstein meant by this and what it implies. _Grazer Philosophische Studien_ 71, 57–86. https://doi.org/10.1163/18756735-07101005.
  * [124] Mühlhölzer, F. (2008). Wittgenstein und der Formalismus. _“Ein Netz vorn Normen”: Wittgenstein und die Mathematik_ , ed. Kroß, M., pp. 107–148 (Parerga).
  * [125] Mühlhölzer, F. (2010). _Braucht die Mathematik eine Grundlegung? Ein Kommentar des Teils III von Wittgensteins ‘Bemerkungen über die Grundlagen der Mathematik’_ (Vittorio Klostermann).
  * [126] Mühlhölzer, F. (2012). Wittgenstein and metamathematics. _Deutsches Jahrbuch Philosophie_ , Band 3, Hg. Stekeler-Weithofer, P., pp. 103–128 (Meiner). https://doi.org/10.5840/djp201235.
  * [127] Muller, F.A. (2004). The implicit definition of the set concept. _Synthese_ 138, 417–451. https://doi.org/10.1023/B:SYNT.0000016439.37687.78.
  * [128] Nederpelt, R., Geuvers, H. (2014). _Type Theory and Formal Proof: An Introduction_ (Cambridge University Press).
  * [129] Nutting, E.S. (undated). The Benacerraf problem of mathematical truth and knowledge. https://iep.utm.edu/benacerraf-problem-of-mathematical-truth-and-knowledge/.
  * [130] Ording, P. (2019). _99 Variations on a Proof_ (Princeton University Press).
  * [131] Panza, M., Sereni, A. (2013). _Plato’s Problem: An Introduction to Mathematical Platonism_ (Palgrave Macmillan).
  * [132] Paseau, A., Pregel, F. (2023). Deductivism in the philosophy of mathematics. _The Stanford Encyclopedia of Philosophy (Fall 2023 Edition)_ , eds. Zalta, E.N., Nodelman, U. https://plato.stanford.edu/archives/fall2023/entries/deductivism-mathematics/.
  * [133] Penrose, R. (2004). _The Road to Reality: A Complete Guide to the Laws of the Universe_ (Jonathan Cape).
  * [134] Pérez-Escobar, J.A. (2022). Showing mathematical flies the way out of foundational bottles: The later Wittgenstein as a forerunner of Lakatos and the philosophy of mathematical practice. _Kriterion–Journal of Philosophy_ 36, 157–178. https://doi.org/10.1515/krt-2021-0041.
  * [135] Pichler, A., Biggs, M.A.R., Szeltner, S.A. (2011). Bibliographie der deutsch- und englischsprachigen Wittgenstein-Ausgaben. http://www.ilwg.eu/files/Wittgenstein_Bibliographie.pdf.
  * [136] Potter, M. (2011). Wittgenstein on mathematics. _The Oxford Handbook of Wittgenstein_ , eds. Kuusela, O., McGinn, M., pp. 122–137. (Oxford University Press).
  * [137] Potter, M. (2020). _The Rise of Analytic Philosophy, 1879–1930: From Frege to Ramsey_ (Routledge).
  * [138] Pyenson, L.R. (1982). Relativity in late-Wilhelminian Germany: The appeal to a pre-established harmony between mathematics and physics. _Archive for History of Exact Sciences_ 27, 137-155. https://www.jstor.org/stable/41133668.
  * [139] Rathjen, M., Sieg, W. (2022). Proof Theory. _The Stanford Encyclopedia of Philosophy (Winter 2022 Edition_ , eds. Zalta, E.N., Nodelman, U. https://plato.stanford.edu/archives/win2022/entries/proof-theory/.
  * [140] Reck, E.H., ed. (2002). _From Frege to Wittgenstein: Perspectives on Early Analytic Philosophy_ (Oxford University Press).
  * [141] Robertson, N., Sanders, D., Seymour, P., Thomas, R. (1997). The four-colour theorem. _Journal of Combinatorial Theory, Series B_ 70, 2–44. https://doi.org/10.1006/jctb.1997.1750.
  * [142] Robinson, R. (1954). _Definition_ (Oxford University Press).
  * [143] Rodych, V. (2018). Wittgenstein’s philosophy of mathematics. _The Stanford Encyclopedia of Philosophy (Spring 2018 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/spr2018/entries/wittgenstein-mathematics/.
  * [144] Rohr, T. (2023). The Frege–Hilbert controversy in context. _Synthese_ 202:12. https://doi.org/10.1007/s11229-023-04196-1.
  * [145] Rush, P. (2022). _Ontology and the Foundations of Mathematics_ (Cambridge University Press).
  * [146] Russell, B. (1914). _Our Knowledge of the External World: As a Field for Scientific Method in Philosophy_ (Open Court). https://www.gutenberg.org/files/37090/37090-h/37090-h.htm.
  * [147] Russell, B., Whitehead, A.N. (1910). _Principia Mathematica, Volume I_ (Cambridge University Press).
  * [148] Sauer, T., Majer, U., eds. (2009). _David Hilbert’s Lectures on the Foundations of Physics 1915–1927_ (Springer).
  * [149] Sauer, T., Majer, U., eds. (2024). _David Hilbert’s Lectures on the Foundations of Physics 1898-1914_ (Springer). To appear.
  * [150] Schlimm, D. (2013). Axioms in mathematical practice. _Philosophia Mathematica_ 21, 37–92. https://doi.org/10.1093/philmat/nks036.
  * [151] Schroeder, S. (2021). _Wittgenstein on Mathematics_ (Routledge).
  * [152] Shanker, S. (1987). _Wittgenstein and the Turning Point in the Philosophy of Mathematics_ (State University of New York Press).
  * [153] Shapiro, A. (1997). _Philosophy of Mathematics: Structure and Ontology_ (Oxford University Press).
  * [154] Shapiro, A. (2005). Categories, structures, and the Frege–Hilbert controversy: The status of meta-mathematics. _Philosophia Mathematica (III)_ 13, 61–77. https://doi.org/10.1093/philmat/nki007.
  * [155] Sieg, W. (2013). _Hilbert’s Programs and Beyond_ (Oxford University Press).
  * [156] Singh, K. (2013). _Is Wiskunde een Spel?_. B.Sc. Thesis, Radboud University. https://www.math.ru.nl/~landsman/Kirti-2023.pdf.
  * [157] Sluga, H. (2006). Family Resemblance. _Grazer Philosophische Studien_ 71, 1–21. https://doi.org/10.1163/18756735-07101002.
  * [158] Soames, S. (2014). _The Analytic Tradition in Philosophy. Volume 1: The Founding Giants_ (Princeton University Press).
  * [159] Steiner, M. (2009). Empirical regularities in Wittgenstein’s philosophy of mathematics. _Philosophia Mathematica_ 17, 1–34. https://doi.org/10.1093/philmat/nkn016.
  * [160] Steingart, A. (2023). _Axiomatics: Mathematical Thought and High Modernism_ (University of Chicago Press).
  * [161] Stenlund, S. (2015). On the origin of symbolic mathematics and its significance for Wittgenstein’s thought. _Nordic Wittgenstein Review_ 4, 7–92. https://doi.org/10.15845/nwr.v4i1.3302.
  * [162] Stillwell, J. (2022). _The Story of Proof: Logic and the History of Mathematics_ (Princeton University Press).
  * [163] Tait, W. (1986). Truth and proof: The Platonism of mathematics. _Synthese_ 69, 341–370. https://doi.org/10.1007/BF00413978.
  * [164] Tait, W. (2001). Beyond the axioms: The question of objectivity in mathematics. _Philosophia Mathematica_ 9, 21–36. https://doi.org/10.1093/philmat/9.1.21.
  * [165] Tapp, C. (2013). _An den Grenzen des Endlichen: Das Hilbertprogramm im Kontext von Formalismus und Finitismus_ (Springer).
  * [166] Thomae, J.K. (1898). _Elementare Theorie der analytischen Functionen einer complexen Veränderlichen. 2 Auflage_ (Halle).
  * [167] Thomae, J.K. (1906a). Gedankenlose Denker. _Jahresbericht der Deutschen Mathematiker-Vereinigung_ 15, 434–438. http://resolver.sub.uni-goettingen.de/purl?PPN37721857X_0015.
  * [168] Thomae, J.K. (1906b). Erklärung. _Jahresbericht der Deutschen Mathematiker-Vereinigung_ 15, 590–592. http://resolver.sub.uni-goettingen.de/purl?PPN37721857X_0015.
  * [169] Thomae, J.K. (1908). Bemerkung zum Aufsatze des Herrn Frege. _Jahresbericht der Deutschen Mathematiker-Vereinigung_ 17, 56. http://resolver.sub.uni-goettingen.de/purl?PPN37721857X_0017.
  * [170] Thurston, W. P. (1994). On proof and progress in mathematics. _Bulletin of the American Mathematical Society_ 30, 161–177. https://doi.org/10.1090/S0273-0979-1994-00502-6.
  * [171] Torretti, R. (1983). _Relativity and Geometry_ (Pergamon Press).
  * [172] Tymoczko, T. (1979). The four-color problem and its philosophical significance. _The Journal of Philosophy_ 76, 57–83. https://doi.org/10.2307/2025976.
  * [173] van Fraassen, B.C. (1980). _The Scientific Image_ (Oxford University Press).
  * [174] van Fraassen, B.C. (1992). _Quantum Mechanics: An Empiricist View_ (Clarendon Press).
  * [175] van Fraassen, B.C. (2008). _Scientific Representation_ (Clarendon Press).
  * [176] Volkert, K., ed. (2015). _David Hilbert: Grundlagen der Geometrie (Festschrift 1899)_ (Springer).
  * [177] von Neumann, J. (1925). Eine Axiomatisierung der Mengenlehre. _Journal für die reine und angewandte Mathematik_ 154, 219–240. https://doi.org/10.1515/crll.1925.154.219. Translation: An axiomatization of set theory. _From Frege to Gödel: A Source Book in Mathematical Logic, 1879–1931_ , ed. van Heijenoort, J., pp. 393–413 (Harvard University Press, 1967).
  * [178] von Neumann, J. (1931). Die formalistische Grundlegung der Mathematik. _Erkenntnis_ 2, 116–121. https://www.jstor.org/stable/20011631. Translation: The formalist foundations of mathematics. _Philosophy of Mathematics: Selected Readings, Second Edition_ , eds. Benacerraf, P., Putnam, H., pp. 61–65 (Cambridge University Press, 1983).
  * [179] von Plato, J. (2017). _The Great Formal Machinery Works: Theories of Deduction and Computation at the Origins of the Digital Age_ (Princeton University Press).
  * [180] von Plato, J. (2018). The Development of Proof Theory. _The Stanford Encyclopedia of Philosophy (Winter 2018 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/win2018/entries/proof-theory-development/.
  * [181] Weiberg, A., Majetschak, S., Hg. (2022). _Wittgenstein Handbuch: Leben – Werk –Wirkung_ (J.B. Metzler).
  * [182] Weyl, H. (1926). Die heutige Erkenntnislage in der Mathematik. _Symposion-Heft_ 3 (Weltkreisverlag, Erlangen). Translation: The current epistemological situation in mathematics, _From Brouwer to Hilbert: The Debate on the Foundations of Mathematics in the 1920s_ , ed. Mancosu, P., pp. 123–142 (Oxford University Press, 1998).
  * [183] Weir, A. (2010). _Truth Through Proof: A Formalist Foundation of Mathematics_ (Oxford University Press).
  * [184] Weir, A. (2014). Informal proof, formal proof, formalism. _The Review of Symbolic Logic_ 9, 23–43. https://doi.org/10.1017/S1755020315000234.
  * [185] Wiedijk. F. (2023). Formalizing 100 Theorems. https://www.cs.ru.nl/~freek/100/index.html.
  * [186] Wikipedia (undated). Classification of finite simple groups. https://en.wikipedia.org/wiki/Classification_of_finite_simple_groups
  * [187] Wittgenstein, L. (1984a). _Philosophische Untersuchungen. Werkausgabe Band 1_ , pp. 225–580 (Suhrkamp). _Wittgenstein’s Nachlass. The Bergen Electronic Edition, Ms-142_. https://wab.uib.no/transform/wab.php?modus=opsjoner. http://www.wittgensteinsource.org/. Translations: _Philosophical Investigations_ , Revised 4th edition by P.M.S. Hacker and Joachim Schulte (Wiley–Blackwell, 2009). _The Collected Works of Ludwig Wittgenstein. Electronic Edition: Philosophical Investigations_. https://www.nlx.com/collections/121.
  * [188] Wittgenstein, L. (1984b). _Wittgenstein und der Wiener Kreis: Gespräche_. _Werkausgabe Band 3_ (Suhrkamp). Translation: _Wittgenstein and the Vienna Circle_ , ed. McGuinness, B.F. (Blackwell, 1967).
  * [189] Wittgenstein, L. (1984c). _Philosophische Grammatik. Werkausgabe Band 4_ (Suhrkamp). _Wittgenstein’s Nachlass. The Bergen Electronic Edition, Ms-113–114_. https://wab.uib.no/transform/wab.php?modus=opsjoner. http://www.wittgensteinsource.org/. Translations: Ludwig Wittgenstein, _The Big Typescript. TS 213_ (Blackwell, 2005). _The Collected Works of Ludwig Wittgenstein. Electronic Edition: Philosophical Grammar_. https://www.nlx.com/collections/121.
  * [190] Wittgenstein, L. (1984d). _Bemerkungen über die Grundlagen der Mathematik. Werkausgabe Band 6_ (Suhrkamp). _Wittgenstein’s Nachlass. The Bergen Electronic Edition, Ms-117, 121–122, 124–127, 164_. https://wab.uib.no/transform/wab.php?modus=opsjoner. http://www.wittgensteinsource.org/. Translations: _Remarks on the Foundations of Mathematics, Second Edition_ (Blackwell, 1968). _The Collected Works of Ludwig Wittgenstein. Electronic Edition: Remarks on the Foundations of Mathematics._ https://www.nlx.com/collections/121.
  * [191] Young, J.O. (2018). The coherence theory of truth. _The Stanford Encyclopedia of Philosophy (Fall 2018 Edition)_ , ed. Zalta, E.N. https://plato.stanford.edu/archives/fall2018/entries/truth-coherence/.
  * [192] Zach. R. (2023). Hilbert’s Program. _The Stanford Encyclopedia of Philosophy (Spring 2023 Edition)_ , ed. Zalta, E.N., Nodelman, U. https://plato.stanford.edu/archives/spr2023/entries/hilbert-program/.


# Model Selection with Gini Indices under Auto-Calibration

Mario V. Wüthrich111RiskLab, Department of Mathematics, ETH Zurich,
<EMAIL_ADDRESS>
(Version of )

###### Abstract

The Gini index does not give a strictly consistent scoring rule in general.
Therefore, maximizing the Gini index may lead to wrong decisions. The main
issue is that the Gini index is a rank-based score that is not calibration-
sensitive. We show that the Gini index allows for strictly consistent scoring
if we restrict to the class of auto-calibrated regression models.

Keywords. Regression model, binary classification, Gini index, Gini score,
consistency, consistent scoring, auto-calibration, Lorenz curve, concentration
curve, cumulative accuracy profile, CAP, receiver operating characteristics,
ROC, area under the curve, AUC, accuracy ratio, Somers’ $D$, forecast-
dominance.

##  1 Introduction

The Gini index (Gini score, accuracy ratio) is a popular tool for model
selection in machine learning, and there are versions of the Gini index that
are used to evaluate actuarial pricing models and financial credit risk
models; see Frees et al. [7, 8], Denuit et al. [3], Engelmann et al. [6] and
Tasche [17]. However, in general, the Gini index does not give a (strictly)
consistent scoring rule; Example 3 of Byrne [1] gives a counterexample.
(Strict) consistency is an important property in model selection because it
ensures that maximizing the Gini index does not lead to a wrong model choice;
see Gneiting [10] and Gneiting–Raftery [11]. The Gini index can be obtained
from Somers’ $D$ [16], which essentially considers Kendall’s $\tau$; see
Newson [14]. Intuitively, this tells us that the Gini index is a rank-based
score that is not calibration-sensitive. The missing piece to make the Gini
index a strictly consistent scoring rule is to restrict it to the class of
auto-calibrated regression models, this is proved in Theorem 4.5, below; for
auto-calibration we refer to Krüger–Ziegel [12], Denuit et al. [2] and Section
7.4.2 of Wüthrich–Merz [18].

Organization. In the next section, we introduce the notion of strictly
consistent scoring rules. In Section 3, we discuss the Gini index as it is
usually used in the machine learning community. In Section 4, we introduce and
discuss the property of having an auto-calibrated regression model
(forecasts), and we prove that the Gini index gives a strictly consistent
scoring rule if we restrict to the class of auto-calibrated regression models.
This makes the maximization of the Gini index a sensible model selection tool
on the class of auto-calibrated regression models. Finally, in Section 5 we
conclude.

##  2 Consistent scoring rules

Let $(Y,\boldsymbol{X})$ be a random tuple on a sufficiently rich probability
space $(\Omega,{\cal A},{\mathbb{P}})$ with real-valued non-negative response
$Y$ having finite mean and with covariates $\boldsymbol{X}$. Denote by ${\cal
F}$ the family of potential distributions of $(Y,\boldsymbol{X})$ being
supported on ${\cal Y}\times{\cal X}$. Let $F_{Y|\boldsymbol{X}}$ be the
conditional distribution of $Y$, given $\boldsymbol{X}$. For any model
$(Y,\boldsymbol{X})\sim F\in{\cal F}$, we consider the conditional mean
functional $T$

$F_{Y|\boldsymbol{X}}~{}\mapsto~{}T(F_{Y|\boldsymbol{X}})=\mu^{\dagger}(\boldsymbol{X})={\mathbb{E}}\left[\left.Y\right|\boldsymbol{X}\right],$

where
$\boldsymbol{X}\mapsto\mu^{\dagger}(\boldsymbol{X})={\mathbb{E}}\left[\left.Y\right|\boldsymbol{X}\right]$
denotes the true regression function of the chosen model. The main task in
regression modeling is to find this unknown true regression function
$\mu^{\dagger}(\cdot)$ from i.i.d. data $(Y_{i},\boldsymbol{X}_{i})$, $1\leq
i\leq n$, having the same distribution as $(Y,\boldsymbol{X})$.

Choose a scoring function $S:{\cal Y}\times{\mathbb{R}}\to{\mathbb{R}}$ giving
us the score
${\mathbb{E}}\left[S\left(Y,\widehat{\mu}(\boldsymbol{X})\right)\right]$ for
regression function $\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ and
$(Y,\boldsymbol{X})\sim F\in{\cal F}$. A scoring rule is obtained by selecting
the argument(s) $\widehat{\mu}^{\star}(\cdot)$ that maximize the score over
the regression functions $\widehat{\mu}(\cdot)$, subject to existence,

$\widehat{\mu}^{\star}(\cdot)~{}\in~{}\underset{\widehat{\mu}(\cdot)}{\arg\max}~{}{\mathbb{E}}\left[S\left(Y,\widehat{\mu}(\boldsymbol{X})\right)\right],$
(2.1)

under the given model choice $(Y,\boldsymbol{X})\sim F\in{\cal F}$.

A scoring rule is called consistent on ${\cal F}$ for the conditional mean
functional $T$, if for any model $(Y,\boldsymbol{X})\sim F\in{\cal F}$ with
conditional distributions $F_{Y|\boldsymbol{X}}$ of $Y$, given
$\boldsymbol{X}$, we have $S(Y,T(F_{Y|{\boldsymbol{X}}}))\in
L^{1}({\mathbb{P}})$, and for any regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ with
$S(Y,\widehat{\mu}(\boldsymbol{X}))\in L^{1}({\mathbb{P}})$ we have

${\mathbb{E}}\left[S\left(Y,T(F_{Y|{\boldsymbol{X}}})\right)\right]\geq{\mathbb{E}}\left[S\left(Y,\widehat{\mu}(\boldsymbol{X})\right)\right].$
(2.2)

A scoring rule is called strictly consistent on ${\cal F}$ for the conditional
mean functional $T$, if it is consistent on ${\cal F}$, and if an identity in
(2.2) holds if and only if
$\widehat{\mu}(\boldsymbol{X})=T(F_{Y|\boldsymbol{X}})=\mu^{\dagger}(\boldsymbol{X})$,
a.s.

###### Remarks 2.1

  * •

Strict consistency implies that the true regression function
$\mu^{\dagger}(\cdot)$ is the unique maximizer in (2.1), and it can be
estimated by score maximization (assuming it is contained in the set over
which we optimize, which we generally do). Empirically, we then consider for
i.i.d. data $(Y_{i},\boldsymbol{X}_{i})$, $1\leq i\leq n$,

$\underset{\widehat{\mu}(\cdot)}{\arg\max}~{}\frac{1}{n}\sum_{i=1}^{n}S\left(Y_{i},\widehat{\mu}(\boldsymbol{X}_{i})\right),$

where we still need to ensure that we can exchange the limit $n\to\infty$ and
the $\arg\max$-operator to asymptotically select the true regression function
$\mu^{\dagger}(\cdot)$ under strict consistency.

  * •

Formula (2.2) states unconditional consistency as we average over the
distribution of $\boldsymbol{X}$. For conditional consistency (in
$\boldsymbol{X}$) and its relation to the unconditional version we refer to
Section 2.2 in Dimitriadis et al. [5]. A point prediction version of
consistency is given in Definition 1 in Gneiting [10].

  * •

For scoring rule (2.1) we consider a maximization. By a sign switch we can
turn this into a minimization problem, and in that case we rather speak about
expected loss minimization.

  * •

Typically, we restrict (2.1)-(2.2) to smaller classes of regression functions
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$. In the sequel, we will
require continuity for these smaller classes, and, further below, we require
the auto-calibration property. This requires that the true regression function
$\mu^{\dagger}(\cdot)$ has this continuity, auto-calibration it will satisfy
automatically, see Lemma 4.1, below.

##  3 The Gini index in machine learning

In the sequel we assume $\widehat{\mu}(\boldsymbol{X})$ to have a continuous
distribution $F_{\widehat{\mu}(\boldsymbol{X})}$ for all
$(Y,\boldsymbol{X})\sim F\in{\cal F}$ and for any considered regression
function $\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$. This implies
$F_{\widehat{\mu}(\boldsymbol{X})}(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha))=\alpha$
for all $\alpha\in(0,1)$, and with $F_{\widehat{\mu}(\boldsymbol{X})}^{-1}$
denoting the left-continuous generalized inverse of
$F_{\widehat{\mu}(\boldsymbol{X})}$.

In machine learning (ML) one considers the cumulative accuracy profile (CAP)
defined by

$\alpha\in(0,1)\quad\mapsto\quad{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)~{}=~{}\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})>F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right]~{}\in~{}[0,1].$

In actuarial science, the CAP is also called concentration curve (up to sign
switches), see Denuit–Trufin [4]. The CAP measures a rank-based correlation
between the prediction $\widehat{\mu}(\boldsymbol{X})$ and the response $Y$.

The Gini index (Gini score, Gini ratio, Gini coefficient, accuracy ratio) in
ML is defined by

$G^{\rm ML}_{Y,\widehat{\mu}(\boldsymbol{X})}=\frac{\int_{0}^{1}{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)\,d\alpha-1/2}{\int_{0}^{1}{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1/2},$ (3.1)

where we additionally assume that $Y$ has an (unconditional) continuous
distribution $F_{Y}$. For a geometric interpretation see Figure 1 (lhs) and
formula (4.3), below.

###### Remarks 3.1

  * •

The denominator in (3.1) does not use the regression function
$\widehat{\mu}(\cdot)$, i.e., it has no impact on model selection by
maximizing the Gini index $G^{\rm ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$ over
$\widehat{\mu}(\cdot)$. Hence, for scoring we can focus on the term in the
enumerator

$\displaystyle\int_{0}^{1}{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)\,d\alpha$ $\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\int_{0}^{1}\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})>F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\,d\alpha\right]$
(3.2) $\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\,{\mathbb{P}}\left[\left.F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(U)<\widehat{\mu}(\boldsymbol{X})\right|\widehat{\mu}(\boldsymbol{X})\right]\right]$
$\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[YF_{\widehat{\mu}(\boldsymbol{X})}(\widehat{\mu}(\boldsymbol{X}))\right],$

for an independent $(0,1)$-uniform random variable $U$ and where we use
continuity of $F_{\widehat{\mu}(\boldsymbol{X})}$. This shows that the Gini
index in ML is not calibration-sensitive because
$F_{\widehat{\mu}(\boldsymbol{X})}(\widehat{\mu}(\boldsymbol{X}))$ has a
$(0,1)$-uniform distribution, i.e., the specific distribution of
$\widehat{\mu}(\boldsymbol{X})$ does not matter, but only its correlation with
$Y$ matters.

  * •

Since typically the true data model $(Y,\boldsymbol{X})\sim F$ is not known,
the Gini index in ML (3.1) is replaced by an empirical version

$\widehat{G}^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}=\frac{\int_{0}^{1}\widehat{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)\,d\alpha-1/2}{\int_{0}^{1}\widehat{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1/2}~{}\leq~{}1,$ (3.3)

where we set

$\widehat{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)=\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{n}Y_{i}\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X}_{i})>\widehat{\mu}\left(\boldsymbol{X}_{\left(\lceil(1-\alpha)n\rceil\right)}\right)\right\\}},$
(3.4)

for i.i.d. data $(Y_{i},\boldsymbol{X}_{i})$, $1\leq i\leq n$, having the same
distribution as $(Y,\boldsymbol{X})$, and for order statistics
$\widehat{\mu}(\boldsymbol{X}_{(1)})<\widehat{\mu}(\boldsymbol{X}_{(2)})<\ldots<\widehat{\mu}(\boldsymbol{X}_{(n)})$;
note that by assumption the distribution of $\widehat{\mu}(\boldsymbol{X})$ is
continuous which implies that all observations
$\widehat{\mu}(\boldsymbol{X}_{i})$ are mutually different for $1\leq i\leq
n$, and we have a strict ordering in the order statistics.

  * •

Let us further comment on (3.4). First, if we mirror the CAP at the diagonal
we have

$\displaystyle{\rm CAP}^{-}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)$
$\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})\leq
F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha)\right\\}}\right]$
$\displaystyle=$ $\displaystyle
1-\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})>F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha)\right\\}}\right]~{}=~{}1-{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(1-\alpha).$

For an empirical version of the mirrored CAP we replace the above expression
by

$\displaystyle\widehat{\rm CAP}^{-}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)$
$\displaystyle=$
$\displaystyle\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{n}Y_{i}\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X}_{i})\leq\widehat{F}_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha)\right\\}}$
$\displaystyle=$ $\displaystyle
1-\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{n}Y_{i}\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X}_{i})>\widehat{\mu}(\boldsymbol{X}_{(\lceil\alpha
n\rceil)})\right\\}},$

where in the last indicator we use the empirical distribution, for
$m\in{\mathbb{R}}$ and $\alpha\in(0,1)$,

$\widehat{F}_{\widehat{\mu}(\boldsymbol{X})}(m)=\frac{1}{n}\,\sum_{i=1}^{n}\mathds{1}_{\\{\widehat{\mu}(\boldsymbol{X}_{i})\leq
m\\}}\qquad\text{ and
}\qquad\widehat{F}^{-1}_{\widehat{\mu}(\boldsymbol{X})}(\alpha)=\widehat{\mu}(\boldsymbol{X}_{(\lceil\alpha
n\rceil)}).$

This justifies the choice in (3.4). Similarly, we have for the denominator in
(3.3)

$\widehat{\rm
CAP}^{-}_{Y,Y}(\alpha)=\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{n}Y_{i}\,\mathds{1}_{\left\\{Y_{i}\,\leq\,Y_{(\lceil\alpha
n\rceil)}\right\\}}~{}\stackrel{{\scriptstyle(*)}}{{=}}~{}\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{\lceil\alpha
n\rceil}Y_{(i)},$ (3.6)

for the identity $\stackrel{{\scriptstyle(*)}}{{=}}$ to hold for any
$\alpha\in(0,1)$, we need to assume that we have a strict ordering
$Y_{(1)}<Y_{(2)}<\ldots<Y_{(n)}$, i.e., that there are no ties in the
observations $(Y_{i})_{1\leq i\leq n}$, which is the case because $Y$ was
assumed to have a continuous distribution $F_{Y}$. This then motivates to set

$\displaystyle\widehat{\rm CAP}_{Y,Y}(\alpha)$ $\displaystyle=$ $\displaystyle
1-\widehat{\rm
CAP}^{-}_{Y,Y}(1-\alpha)=\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{n}Y_{i}\,\mathds{1}_{\left\\{Y_{i}\,>\,Y_{(\lceil(1-\alpha)n\rceil)}\right\\}}$
(3.7) $\displaystyle\stackrel{{\scriptstyle(*)}}{{=}}$
$\displaystyle\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=\lceil(1-\alpha)n\rceil+1}^{n}Y_{(i)}.$

If we have a perfect joint ordering between $(Y_{i})_{1\leq i\leq n}$ and
$(\widehat{\mu}(\boldsymbol{X}_{i}))_{1\leq i\leq n}$, the upper bound in
(3.3) is attained, see (3.4) and (3.7). This is the motivation for the scaling
in (3.1).

In the definition of the Gini index in ML (3.1) we have assumed that $Y$ has a
continuous distribution $F_{Y}$. This is not the case for discrete responses
$Y$. Therefore, in the discrete case we need to replace the denominator in
(3.1) by a different object. For illustrative purposes we show the binary
classification case in the next example.

######  Example 3.2 (binary classification)

We consider a binary classification example with true regression function

$\boldsymbol{X}~{}\mapsto~{}p^{\dagger}(\boldsymbol{X})={\mathbb{E}}\left[\left.Y\right|\boldsymbol{X}\right]={\mathbb{P}}\left[\left.Y=1\right|\boldsymbol{X}\right]~{}\in~{}(0,1).$

That is, $Y$ is conditionally Bernoulli distributed, given $\boldsymbol{X}$,
with probability $p^{\dagger}(\boldsymbol{X})\in(0,1)$ and range ${\cal
Y}=\\{0,1\\}$. In this case the CAP for a regression function
$\boldsymbol{X}\mapsto\widehat{p}(\boldsymbol{X})$ with continuous
distribution $F_{\widehat{p}(\boldsymbol{X})}$ is for $\alpha\in(0,1)$ given
by

$\displaystyle{\rm CAP}_{Y,\widehat{p}(\boldsymbol{X})}(\alpha)$
$\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\,\mathds{1}_{\left\\{\widehat{p}(\boldsymbol{X})>F_{\widehat{p}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right]$
$\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{P}}[Y=1]}\,{\mathbb{E}}\left[\mathds{1}_{\left\\{Y=1,\,\widehat{p}(\boldsymbol{X})>F_{\widehat{p}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right]$
$\displaystyle=$
$\displaystyle{\mathbb{P}}\left[\left.\widehat{p}(\boldsymbol{X})>F_{\widehat{p}(\boldsymbol{X})}^{-1}(1-\alpha)\right|Y=1\right]$
$\displaystyle=$ $\displaystyle
1-F_{\widehat{p}(\boldsymbol{X})|Y=1}\left(F_{\widehat{p}(\boldsymbol{X})}^{-1}(1-\alpha)\right).$

This corresponds to formula (5.2) in Tasche [17].

For the Gini index in ML we need to calculate the denominator of (3.1).
However, this formula only applies for a continuous distribution $F_{Y}$ of
$Y$. In the case of a discrete distribution of $Y$ we need to modify (3.1).
Starting from the right-hand side of (3.6), we define the empirical function
in the discrete case by

$\alpha\in(0,1)~{}\mapsto~{}\widehat{\rm
CAP}^{-}_{Y,Y}(\alpha)~{}=~{}\frac{1}{\frac{1}{n}\sum_{i=1}^{n}Y_{i}}\,\frac{1}{n}\sum_{i=1}^{\lceil\alpha
n\rceil}Y_{(i)},$

for i.i.d. data $(Y_{i},\boldsymbol{X}_{i})$, $1\leq i\leq n$. In the
Bernoulli case, this function is identically equal to zero up to $\alpha\leq
1-\sum_{i=1}^{n}Y_{i}/n$, these describes the number of zeros among the
observations $(Y_{i})_{1\leq i\leq n}$, and afterwards it increases to 1.
Since this increase is only described on the discrete grid with span $1/n$, we
linearly interpolate between these points. This provides a straight line
between $1-\sum_{i=1}^{n}Y_{i}/n$ and 1 with slope $n/\sum_{i=1}^{n}Y_{i}$.
Under this linear interpolation, we get the area (integral)

$\int_{0}^{1}\widehat{\rm
CAP}^{-}_{Y,Y}(\alpha)\,d\alpha=\frac{1}{2n}\,\sum_{i=1}^{n}Y_{i}.$

By the law of large numbers, the latter converges to
$p^{\dagger}/2={\mathbb{E}}[p^{\dagger}(\boldsymbol{X})]/2={\mathbb{E}}[Y]/2$,
a.s., as $n\to\infty$. This motivates in the (discrete) binary classification
case the following definition of the Gini index in ML

$G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}=\frac{1/2-\int_{0}^{1}F_{\widehat{p}(\boldsymbol{X})|Y=1}\left(F_{\widehat{p}(\boldsymbol{X})}^{-1}(1-\alpha)\right)d\alpha}{(1-p^{\dagger})/2}.$
(3.8)

In the binary classification case, the CAP can be related to the receiver
operating characteristics (ROC) curve. The area under the curve (AUC) of the
ROC curve has a one-to-one relationship to the Gini index in ML (3.8) in the
Bernoulli case, we refer to Section 5 in Tasche [17]. We mention this because
the ML community more frequently uses the AUC than the Gini index for model
selection.

In general, in the discrete case we replace the integral in the denominator in
(3.1) by the term

$\frac{1}{4{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[\left|Y-\widetilde{Y}\right|\right],$
(3.9)

where $\widetilde{Y}$ is an independent copy of $Y$. This latter quantity
(3.9) can be calculated for any distribution $F_{Y}$ of $Y$, and in the
continuous case we precisely receive the denominator in (3.1). The binary
classification case (3.9) provides us with $(1-p^{\dagger})/2$ which gives
(3.8). $\blacksquare$

##  4 Auto-calibration and consistency of the Gini index

Let $(Y,\boldsymbol{X})\sim F$. A regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ is auto-calibrated for
$Y$ if, a.s.,

$\widehat{\mu}(\boldsymbol{X})={\mathbb{E}}\left[Y\left|\widehat{\mu}(\boldsymbol{X})\right]\right..$

Auto-calibration is an important property in insurance pricing, as it implies
that every cohort of insurance policies paying the same price
$\widehat{\mu}(\boldsymbol{X})$ is in average self-financing, because the
price $\widehat{\mu}(\boldsymbol{X})$ exactly covers the expected claim $Y$ of
that cohort. I.e., we do not have any systematic cross-financing between the
price cohorts. This is the core of risk classification in insurance. It also
implies unbiasedness on the portfolio level

${\mathbb{E}}\left[\widehat{\mu}(\boldsymbol{X})\right]={\mathbb{E}}\left[Y\right],$
(4.1)

which is a minimal requirement in insurance pricing. Typically, there are many
auto-calibrated regression functions $\widehat{\mu}(\boldsymbol{X})$ for $Y$,
i.e., there are many systems of self-financing pricing cohorts.

###### Lemma 4.1

The true regression function
$\boldsymbol{X}\mapsto\mu^{\dagger}(\boldsymbol{X})={\mathbb{E}}[Y|\boldsymbol{X}]$
is auto-calibrated for $Y$, and it strictly dominates in convex order any
other auto-calibrated regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ for $Y$.

Proof. To prove auto-calibration of $\mu^{\dagger}$ we apply the tower
property to the $\sigma$-algebras
$\sigma(\mu^{\dagger}(\boldsymbol{X}))\subset\sigma(\boldsymbol{X})$ which
gives, a.s.,

${\mathbb{E}}\left[Y\left|\mu^{\dagger}(\boldsymbol{X})\right]\right.={\mathbb{E}}\left[{\mathbb{E}}\left[Y\left|\boldsymbol{X}\right]\right.\left|\mu^{\dagger}(\boldsymbol{X})\right]\right.={\mathbb{E}}\left[\mu^{\dagger}(\boldsymbol{X})\left|\mu^{\dagger}(\boldsymbol{X})\right]\right.=\mu^{\dagger}(\boldsymbol{X}).$

For any convex function $\psi$, auto-calibration, the tower property for
$\sigma(\widehat{\mu}(\boldsymbol{X}))\subset\sigma(\boldsymbol{X})$ and
Jensen’s inequality give

$\displaystyle{\mathbb{E}}\left[\psi\left(\widehat{\mu}(\boldsymbol{X})\right)\right]$
$\displaystyle=$
$\displaystyle{\mathbb{E}}\left[\psi\left({\mathbb{E}}\left[Y\left|\widehat{\mu}(\boldsymbol{X})\right]\right.\right)\right]~{}=~{}{\mathbb{E}}\left[\psi\left({\mathbb{E}}\left[{\mathbb{E}}\left[Y\left|\boldsymbol{X}\right]\right.\left|\widehat{\mu}(\boldsymbol{X})\right]\right.\right)\right]~{}=~{}{\mathbb{E}}\left[\psi\left({\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})\right|\widehat{\mu}(\boldsymbol{X})\right]\right)\right]$
$\displaystyle\leq$
$\displaystyle{\mathbb{E}}\left[{\mathbb{E}}\left[\left.\psi\left(\mu^{\dagger}(\boldsymbol{X})\right)\right|\widehat{\mu}(\boldsymbol{X})\right]\right]~{}=~{}{\mathbb{E}}\left[\psi\left(\mu^{\dagger}(\boldsymbol{X})\right)\right],$

whenever these exist. This proves that $\mu^{\dagger}$ dominates in convex
order any other auto-calibrated regression function $\widehat{\mu}$ for $Y$.
Assume that there exists an auto-calibrated regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ for $Y$ such that for any
convex function $\psi$ we have an equality in the previous calculation,
whenever these exist. This implies that $\mu^{\dagger}(\boldsymbol{X})$ is
$\sigma(\widehat{\mu}(\boldsymbol{X}))$-measurable. Auto-calibration and the
tower property for
$\sigma(\widehat{\mu}(\boldsymbol{X}))\subset\sigma(\boldsymbol{X})$ then
provide, a.s.,

$\widehat{\mu}(\boldsymbol{X})={\mathbb{E}}\left[Y\left|\widehat{\mu}(\boldsymbol{X})\right]\right.={\mathbb{E}}\left[{\mathbb{E}}\left[Y\left|\boldsymbol{X}\right]\right.\left|\widehat{\mu}(\boldsymbol{X})\right]\right.={\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})\right|\widehat{\mu}(\boldsymbol{X})\right]=\mu^{\dagger}(\boldsymbol{X}).$

This proves the statement of strict convex order. $\Box$

The next proposition is a consequence of Lemma 4.1 and of Theorem 3.1 in
Krüger–Ziegel [12].

###### Proposition 4.2

The true regression function
$\boldsymbol{X}\mapsto\mu^{\dagger}(\boldsymbol{X})$ forecast-dominates any
auto-calibrated regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ for $Y$ meaning that

${\mathbb{E}}\left[-D_{\psi}(Y,\mu^{\dagger}(\boldsymbol{X}))\right]\geq{\mathbb{E}}\Big{[}-D_{\psi}(Y,\widehat{\mu}(\boldsymbol{X}))\Big{]},$

for any convex function $\psi$ where the above exists, and with Bregman
divergence given by

$D_{\psi}(y,m)=\psi(y)-\psi(m)-\psi^{\prime}(m)(y-m)~{}\geq~{}0,$

for $y,m\in{\mathbb{R}}$ and $\psi^{\prime}$ is a (sub-)gradient of the convex
function $\psi$.

Proposition 4.2 says that every negative Bregman divergence provides a
consistent scoring rule (2.2) for the conditional mean regression functional
$T$ under auto-calibration for $Y$. This statement motivates the common
practice in model selection of minimizing (out-of-sample) deviance losses, as
deviance losses are special cases of Bregman divergences; see Chapters 2 and 4
in Wüthrich–Merz [18]. For more information on this topic we refer to
Krüger–Ziegel [12], Theorem 7 in Gneiting [10] and Savage [15], the latter two
references state that Bregman divergences provide the only strictly consistent
scoring functions for mean estimation.

The definition of the Gini index [9] in economics slightly differs from the ML
version (3.1). Assume $F_{\widehat{\mu}(\boldsymbol{X})}$ is a continuous
distribution. It is then based on the Lorenz curve [13] given by

$\alpha\in(0,1)\quad\mapsto\quad
L_{{\widehat{\mu}(\boldsymbol{X})}}\left(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha)\right)~{}=~{}\frac{1}{{\mathbb{E}}[\widehat{\mu}(\boldsymbol{X})]}\,{\mathbb{E}}\left[\widehat{\mu}(\boldsymbol{X})\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})\leq
F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha)\right\\}}\right]~{}\in~{}[0,1].$

Note that we have the property ${\rm
CAP}^{-}_{\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{X})}=L_{{\widehat{\mu}(\boldsymbol{X})}}(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha))$,
see (• ‣ 3.1).

The Gini index in economics has many (equivalent)222For an equivalence in
(4.2) we need that $F_{\widehat{\mu}(\boldsymbol{X})}$ is continuous,
otherwise one should choose the term on the right-hand side as the definition
of the Gini index in economics. definitions, we use the following two

$G^{\rm
eco}_{\widehat{\mu}(\boldsymbol{X})}=1-2\int_{0}^{1}L_{{\widehat{\mu}(\boldsymbol{X})}}\left(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\alpha)\right)d\alpha=\frac{1}{2{\mathbb{E}}[\widehat{\mu}(\boldsymbol{X})]}\,{\mathbb{E}}\Big{[}\Big{|}\widehat{\mu}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\Big{|}\Big{]},$
(4.2)

where $\widehat{\mu}(\boldsymbol{Z})$ is an independent copy of
$\widehat{\mu}(\boldsymbol{X})$. The first definition in (4.2) is based on a
continuous distribution $F_{\widehat{\mu}(\boldsymbol{X})}$, whereas the
second one can be used for any distribution
$F_{\widehat{\mu}(\boldsymbol{X})}$, we also refer to (3.9).

Figure 1: (lhs) cumulative accuracy profile (CAP) and (rhs) Lorenz curve.

There are three differences between the Gini index in ML and the one in
economics, see Figure 1: (i) $G^{\rm eco}_{\widehat{\mu}(\boldsymbol{X})}$
considers a mirrored version of the curves compared to $G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$; (ii) $G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$ depends on $Y$ and
$\widehat{\mu}(\boldsymbol{X})$, $G^{\rm eco}_{\widehat{\mu}(\boldsymbol{X})}$
only depends on $\widehat{\mu}(\boldsymbol{X})$; (iii) scalings are different
leading to areas B and C, respectively, in Figure 1. The two Gini indices are
geometrically obtained by, see Figure 1,

$G^{\rm ML}_{Y,\widehat{\mu}(\boldsymbol{X})}=\frac{\text{area}({\rm
A})}{\text{area}({\rm A}+{\rm B})}\quad\text{ and }\quad G^{\rm
eco}_{\widehat{\mu}(\boldsymbol{X})}=\frac{\text{area}({\rm
A})}{\text{area}({\rm A}+{\rm C})}=2\,\text{area}({\rm
A})=1-2\,\text{area}({\rm C}).$ (4.3)

Property 3.1 of Denuit–Trufin [4] gives the following nice result.

###### Proposition 4.3

Under auto-calibration of the regression function
$\boldsymbol{X}\to\widehat{\mu}(\boldsymbol{X})$ for $Y$ we have the identity
${\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)=1-L_{\widehat{\mu}(\boldsymbol{X})}(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha))$
for all $\alpha\in(0,1)$.

Proof. Using the tower property, auto-calibration of $\widehat{\mu}$ for $Y$
and unbiasedness (4.1) give us

$\displaystyle{\rm CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)$
$\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[Y\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})>F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right]~{}=~{}\frac{1}{{\mathbb{E}}[Y]}\,{\mathbb{E}}\left[{\mathbb{E}}\left[\left.Y\right|\widehat{\mu}(\boldsymbol{X})\right]\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})>F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right]$
$\displaystyle=$
$\displaystyle\frac{1}{{\mathbb{E}}[\widehat{\mu}(\boldsymbol{X})]}\,{\mathbb{E}}\left[\widehat{\mu}(\boldsymbol{X})\,\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})>F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right]~{}=~{}1-\frac{1}{{\mathbb{E}}[\widehat{\mu}(\boldsymbol{X})]}\,{\mathbb{E}}\left[\widehat{\mu}(\boldsymbol{X})\mathds{1}_{\left\\{\widehat{\mu}(\boldsymbol{X})\leq
F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha)\right\\}}\right].$

This proves the claim. $\Box$

Thus, under auto-calibration for $Y$, the CAP and the Lorenz curve coincide
(up to mirroring/sign switching). This gives us the following corollary.

###### Corollary 4.4

Under auto-calibration of the regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ for $Y$ we have for the
Gini indices

$G^{\rm ML}_{Y,\widehat{\mu}(\boldsymbol{X})}=\frac{G^{\rm
eco}_{\widehat{\mu}(\boldsymbol{X})}}{2\int_{0}^{1}{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1}.$ (4.4)

Proof. Proposition 4.3 gives us for the Gini index in ML

$\displaystyle G^{\rm ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$ $\displaystyle=$
$\displaystyle\frac{\int_{0}^{1}{\rm
CAP}_{Y,\widehat{\mu}(\boldsymbol{X})}(\alpha)\,d\alpha-1/2}{\int_{0}^{1}{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1/2}~{}=~{}\frac{1/2-\int_{0}^{1}L_{\widehat{\mu}(\boldsymbol{X})}(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha))\,d\alpha}{\int_{0}^{1}{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1/2}$ $\displaystyle=$
$\displaystyle\frac{1-2\int_{0}^{1}L_{\widehat{\mu}(\boldsymbol{X})}(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(1-\alpha))\,d\alpha}{2\int_{0}^{1}{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1}~{}=~{}\frac{1-2\int_{0}^{1}L_{\widehat{\mu}(\boldsymbol{X})}(F_{\widehat{\mu}(\boldsymbol{X})}^{-1}(\beta))\,d\beta}{2\int_{0}^{1}{\rm
CAP}_{Y,Y}(\alpha)\,d\alpha-1},$

where the last step uses the change of variable $\alpha\mapsto\beta=1-\alpha$.
This proves the claim. $\Box$

This says that under auto-calibration for the response both Gini indices (the
ML score and the version in economics) provide the same scoring rule because
the (positive) denominator333Note that the denominator in (4.4) is positive
for every non-deterministic $Y$. This follows from the fact that the
denominator is equal to twice (3.9) which is positive unless $Y$ is
deterministic. in (4.4) does not depend on the specific choice of the
regression function $\widehat{\mu}(\cdot)$. Moreover, the same arguments apply
to the Gini indices in non-continuous cases, e.g., in the binary
classification (Bernoulli) case (3.8).

###### Theorem 4.5

The true regression function
$\boldsymbol{X}\mapsto\mu^{\dagger}(\boldsymbol{X})$ maximizes the Gini index
(in ML) among all auto-calibrated regression functions
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$ for $Y$, i.e., $G^{\rm
ML}_{Y,\mu^{\dagger}(\boldsymbol{X})}>G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$ unless
$\widehat{\mu}(\boldsymbol{X})=\mu^{\dagger}(\boldsymbol{X})$, a.s.

Proof. Conditionally, given $\widehat{\mu}(\boldsymbol{Z})$,
$m\mapsto|m-\widehat{\mu}(\boldsymbol{Z})|$ is a convex function in
$m\in{\mathbb{R}}$. Using formula (4.2), independence between
$\widehat{\mu}(\boldsymbol{X})$ and $\widehat{\mu}(\boldsymbol{Z})$ in (4.2)
and Lemma 4.1 we obtain inequality, a.s.,

${\mathbb{E}}\left[\left.\left|\widehat{\mu}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\,\right|\widehat{\mu}(\boldsymbol{Z})\right]\leq{\mathbb{E}}\left[\left.\left|\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\,\right|\widehat{\mu}(\boldsymbol{Z})\right],$

where $\mu^{\dagger}(\boldsymbol{X})$ is independent of
$\widehat{\mu}(\boldsymbol{Z})$. Using the tower property, applying the same
argument to the exchanged role of $\mu^{\dagger}(\boldsymbol{X})$ and
$\widehat{\mu}(\boldsymbol{Z})$, using unbiasedness (4.1) and using Corollary
4.4 provides $G^{\rm ML}_{Y,\mu^{\dagger}(\boldsymbol{X})}\geq G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$.

Assume there exists an auto-calibrated regression function $\widehat{\mu}$ for
$Y$ such that $G^{\rm ML}_{Y,\mu^{\dagger}(\boldsymbol{X})}=G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$. Using auto-calibration of
$\widehat{\mu}$ for $Y$ and the tower property, we receive for
${\mathbb{P}}$-a.e. $\omega\in\Omega$

$\widehat{\mu}(\boldsymbol{X})(\omega)={\mathbb{E}}\left[Y\left|\widehat{\mu}(\boldsymbol{X})\right]\right.(\omega)={\mathbb{E}}\left[{\mathbb{E}}\left.\left[Y\right|\boldsymbol{X}\right]\left|\widehat{\mu}(\boldsymbol{X})\right]\right.(\omega)={\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})\right|\widehat{\mu}(\boldsymbol{X})\right](\omega).$
(4.5)

Denote by $\Omega_{1}\subset\Omega$ a set of full measure 1 on which (4.5)
holds. On $\Omega_{1}$, the predictor $\widehat{\mu}(\boldsymbol{X})$ is
between the conditional essential infimum and supremum of
$\mu^{\dagger}(\boldsymbol{X})$, given $\widehat{\mu}(\boldsymbol{X})$,
because it corresponds to the conditional expectation of
$\mu^{\dagger}(\boldsymbol{X})$, given $\widehat{\mu}(\boldsymbol{X})$.
Consider the case of sample points $\omega\in\Omega_{1}$ where the conditional
essential infimum and supremum of $\mu^{\dagger}(\boldsymbol{X})$, given
$\widehat{\mu}(\boldsymbol{X})$, do not coincide, and denote the corresponding
set of sample points by $\Omega_{2}\subset\Omega_{1}$. On $\Omega_{2}$, the
predictor $\widehat{\mu}(\boldsymbol{X})$ is strictly between the conditional
essential infimum and supremum of $\mu^{\dagger}(\boldsymbol{X})$, given
$\widehat{\mu}(\boldsymbol{X})$, due to the conditional expectation property
(4.5). We have using (4.5) and independence between
$\widehat{\mu}(\boldsymbol{X})$ and $\widehat{\mu}(\boldsymbol{Z})$

$\displaystyle{\mathbb{E}}\left[\left|\widehat{\mu}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\right]$
$\displaystyle=$
$\displaystyle{\mathbb{E}}\left[\left|{\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})\right|\widehat{\mu}(\boldsymbol{X})\right]-\widehat{\mu}(\boldsymbol{Z})\right|\right]~{}=~{}{\mathbb{E}}\left[\left|{\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right]\right|\right]$
(4.6) $\displaystyle=$
$\displaystyle{\mathbb{E}}\left[\left(\mathds{1}_{\Omega_{2}}+\mathds{1}_{\Omega_{2}^{c}}\right)\left|{\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right]\right|\right].$

We calculate the first term on the right-hand side of (4.6)

${\mathbb{E}}\left[\mathds{1}_{\Omega_{2}}\left|{\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right]\right|\,\right]=\int_{\Omega_{2}}\left(\int_{\Omega}\left|{\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right]\right|(\omega,\widetilde{\omega})\,d{\mathbb{P}}(\widetilde{\omega})\right)d{\mathbb{P}}(\omega).$

We study the inner integral for fixed sample point $\omega\in\Omega_{2}$.
Jensen’s inequality gives us

$\int_{\Omega}\left|{\mathbb{E}}\left[\left.\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right]\right|(\omega,\widetilde{\omega})\,d{\mathbb{P}}(\widetilde{\omega})~{}<~{}\int_{\Omega}{\mathbb{E}}\left[\left.\left|\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\,\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right](\omega,\widetilde{\omega})\,d{\mathbb{P}}(\widetilde{\omega}),$
(4.7)

where we receive a strict inequality for $\omega\in\Omega_{2}$ because of the
following items: (1) on $\Omega_{2}$, $\mu^{\dagger}(\boldsymbol{X})$ is non-
deterministic, conditionally given $\widehat{\mu}(\boldsymbol{X})$, (2)
$m\mapsto|m-\widehat{\mu}(\boldsymbol{Z})|$ is a convex function, (3)
$\widehat{\mu}(\boldsymbol{Z})$ has the same distribution (and support) as
$\widehat{\mu}(\boldsymbol{X})$, and (4) $\widehat{\mu}(\boldsymbol{Z})$ and
$(\mu^{\dagger}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{X}))$ are
independent. Items (1)-(4) imply that on a set of positive
${\mathbb{P}}(\widetilde{\omega})$-measure we receive a strict Jensen’s
inequality, because on this set, $\widehat{\mu}(\boldsymbol{Z})$ is strictly
within the conditional essential infimum and supremum of (the non-
deterministic) $\mu^{\dagger}(\boldsymbol{X})$, given
$\widehat{\mu}(\boldsymbol{X})$.

Assume ${\mathbb{P}}[\Omega_{2}]>0$, i.e., strict inequality (4.7) occurs on a
set of positive measure. Applying Jensen’s inequality also to the other term
in (4.6) we receive strict inequality

${\mathbb{E}}\left[\left|\widehat{\mu}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\right]~{}<~{}{\mathbb{E}}\left[{\mathbb{E}}\left[\left.\left|\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\right|\widehat{\mu}(\boldsymbol{X}),\widehat{\mu}(\boldsymbol{Z})\right]\right]={\mathbb{E}}\left[\left|\mu^{\dagger}(\boldsymbol{X})-\widehat{\mu}(\boldsymbol{Z})\right|\right].$

This strict inequality contradicts our assumption $G^{\rm
ML}_{Y,\mu^{\dagger}(\boldsymbol{X})}=G^{\rm
ML}_{Y,\widehat{\mu}(\boldsymbol{X})}$. Therefore,
${\mathbb{P}}[\Omega_{2}]=0$, which implies

${\mathbb{P}}\left[\Omega_{2}^{c}\cap\Omega_{1}\right]={\mathbb{P}}\left[\Omega_{2}^{c}\right]=1.$

On the set $\Omega_{2}^{c}\cap\Omega_{1}$, we have
$\mu^{\dagger}(\boldsymbol{X})=\widehat{\mu}(\boldsymbol{X})$, which proves
the claim. $\Box$

Theorem 4.5 proves that the Gini index gives a strictly consistent scoring
rule on the class of auto-calibrated regression functions that are
$\boldsymbol{X}$-measurable, because the true regression function
$\boldsymbol{X}\mapsto\mu^{\dagger}(\boldsymbol{X})$ maximizes this Gini
index. A bigger Gini index can only be achieved by a larger information set
than the $\sigma$-algebra generated by $\boldsymbol{X}$.

The following proposition generalizes Property 5.1 of Denuit et al. [2], which
gives a method of restoring auto-calibration for a general regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$.

###### Proposition 4.6

Consider a regression function
$\boldsymbol{X}\mapsto\widehat{\mu}(\boldsymbol{X})$. The following regression
function is auto-calibrated for $Y$

$\boldsymbol{X}~{}\mapsto~{}\widehat{\mu}^{\rm(auto)}(\boldsymbol{X})={\mathbb{E}}\left[Y\left|\widehat{\mu}(\boldsymbol{X})\right]\right..$

Proof. Note that $\widehat{\mu}^{\rm(auto)}(\boldsymbol{X})$ is
$\sigma(\widehat{\mu}(\boldsymbol{X}))$-measurable. This implies
$\sigma(\widehat{\mu}^{\rm(auto)}(\boldsymbol{X}))\subset\sigma(\widehat{\mu}(\boldsymbol{X}))$.
Henceforth, using the tower property, a.s.,

${\mathbb{E}}\left[Y\left|\widehat{\mu}^{\rm(auto)}(\boldsymbol{X})\right]\right.={\mathbb{E}}\left[{\mathbb{E}}\left[Y\left|\widehat{\mu}(\boldsymbol{X})\right]\right.\left|\widehat{\mu}^{\rm(auto)}(\boldsymbol{X})\right]\right.={\mathbb{E}}\left[\widehat{\mu}^{\rm(auto)}(\boldsymbol{X})\left|\widehat{\mu}^{\rm(auto)}(\boldsymbol{X})\right]\right.=\widehat{\mu}^{\rm(auto)}(\boldsymbol{X}).$

This completes the proof. $\Box$

##  5 Conclusions

In general, one should not use the Gini index for model selection because it
does not give a strictly consistent scoring rule and, thus, may lead to wrong
decisions. We have shown in Theorem 4.5 that if we restrict Gini index scoring
to the class of auto-calibrated regression functions for the given response,
the Gini index allows for strictly consistent scoring. This also translates to
the binary classification case where the (machine learning version of the)
Gini index has an equivalent formulation in terms of the area under the curve
(AUC) of the receiver operating characteristics (ROC) curve, we refer to
Tasche [17]. We only need to ensure that the binary classification model is
auto-calibrated for the Bernoulli response to receive a strictly consistent
scoring rule from the AUC.

## References

  * [1] Byrne, S. (2016). A note on the use of empirical AUC for evaluating probabilistic forecasts. Electronic Journal of Statistics 10, 380-393.
  * [2] Denuit, M., Charpentier, A., Trufin, J. (2021). Autocalibration and Tweedie-dominance for insurance pricing in machine learning. Insurance: Mathematics & Economics 101/B, 485-497.
  * [3] Denuit, M., Sznajder, D., Trufin, J. (2019). Model selection based on Lorenz and concentration curves, Gini indices and convex order. Insurance: Mathematics & Economics 89, 128-139.
  * [4] Denuit, M., Trufin, J. (2021). Lorenz curve, Gini coefficient, and Tweedie dominance for autocalibrated predictors. LIDAM Discussion Paper ISBA 2021/36.
  * [5] Dimitriadis, T., Fissler, T., Ziegel, J.F. (2020). The efficiency gap. arXiv, 2010.14146.
  * [6] Engelmann, B., Hayden, E., Tasche, D. (2003). Testing rating accuracy. Risk 16/1, 82-86.
  * [7] Frees, E.W., Meyers, G., Cummings, A.D. (2011). Summarizing insurance scores using a Gini index. Journal of the American Statistical Association 106, 1085-1098.
  * [8] Frees, E.W., Meyers, G., Cummings, A.D. (2013). Insurance ratemaking and a Gini index. Journal of Risk and Insurance 81, 335-366.
  * [9] Gini, C. (1912). Variabilità e Mutuabilità. Contributo allo Studio delle Distribuzioni e delle Relazioni Statistiche. C. Cuppini, Bologna.
  * [10] Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association 106/494, 746-762.
  * [11] Gneiting, T., Raftery, A.E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102/477, 359-378.
  * [12] Krüger, F., Ziegel, J.F. (2021). Generic conditions for forecast dominance. Journal of Business & Economics Statistics 39/4, 972-983.
  * [13] Lorenz, M.O. (1905). Methods of measuring the concentration of wealth. Publications of the American Statistical Association 9/70, 209-219.
  * [14] Newson, R. (2002). Parameters behind ”nonparametric” statistics: Kendall’s tau, Somers’ $D$ and median differences. Stata Journal 2/1, 45-64.
  * [15] Savage, L.J. (1971). Elicitable of personal probabilities and expectations. Journal of the American Statistical Association 66/336, 783-810.
  * [16] Somers, R.H. (1962). A new asymmetric measure of association for ordinal variables. American Sociological Review 27/6, 799-811.
  * [17] Tasche, D. (2006). Validation of internal rating systems and PD estimates. arXiv:0606071.
  * [18] Wüthrich, M.V., Merz, M. (2022). Statistical Foundations of Actuarial Learning and its Applications. Springer Actuarial, in press.