The idea that “simplicity is a sign of truth”, and the related “Occam’s razor” principle, stating that, all other things being equal, simpler models should be preferred to more complex ones, have been long discussed in philosophy and science. We explore these ideas in the context of supervised machine learning, namely the branch of artificial intelligence that studies algorithms which balance simplicity and accuracy in order to effectively learn about the features of the underlying domain. Focusing on statistical learning theory, (...) we show that situations exist for which a preference for simpler models provably slows down, instead of favoring, the supervised learning process. Our results shed new light on the relations between simplicity and truth approximation, which are briefly discussed in the context of both machine learning and the philosophy of science. (shrink)

The no-free-lunch theorems promote a skeptical conclusion that all possible machine learning algorithms equally lack justification. But how could this leave room for a learning theory, that shows that some algorithms are better than others? Drawing parallels to the philosophy of induction, we point out that the no-free-lunch results presuppose a conception of learning algorithms as purely data-driven. On this conception, every algorithm must have an inherent inductive bias, that wants justification. We argue that many standard learning algorithms should rather (...) be understood as model-dependent: in each application they also require for input a model, representing a bias. Generic algorithms themselves, they can be given a model-relative justification. (shrink)

The universal optimality theorem for metainduction works for epistemic agents faced with a choice among finitely many prediction methods. Eckhart Arnold and Tom Sterkenburg objected that it breaks...

I evaluate Schurz's proposed meta-inductive justification of induction, a refinement of Reichenbach's pragmatic justification that rests on results from the machine learning branch of prediction with expert advice. My conclusion is that the argument, suitably explicated, comes remarkably close to its grand aim: an actual justification of induction. This finding, however, is subject to two main qualifications, and still disregards one important challenge. The first qualification concerns the empirical success of induction. Even though, I argue, Schurz's argument does not need (...) to spell out what inductive method actually consists in, it does need to postulate that there is something like the inductive or scientific prediction strategy that has so far been significantly more successful than alternative approaches. The second qualification concerns the difference between having a justification for inductive method and for sticking with induction for now. Schurz's argument can only provide the latter. Finally, the remaining challenge concerns the pool of alternative strategies, and the relevant notion of a meta-inductivist's optimality that features in the analytic step of Schurz's argument. Building on the work done here, I will argue in a follow-up paper that the argument needs a stronger dynamic notion of a meta-inductivist's optimality. (shrink)

Roger White (2015) sketches an ingenious new solution to the problem of induction. He argues from the principle of indifference for the conclusion that the world is more likely to be induction- friendly than induction-unfriendly. But there is reason to be skeptical about the proposed indifference-based vindication of induction. It can be shown that, in the crucial test cases White concentrates on, the assumption of indifference renders induction no more accurate than random guessing. After discussing this result, the paper explains (...) why the indifference-based argument seemed so compelling, despite ultimately being unsound. (shrink)

Ravit Dotan argues that a No Free Lunch theorem from machine learning shows epistemic values are insufficient for deciding the truth of scientific hypotheses. She argues that NFL shows that the best case accuracy of scientific hypotheses is no more than chance. Since accuracy underpins every epistemic value, non-epistemic values are needed to assess the truth of scientific hypotheses. However, NFL cannot be coherently applied to the problem of theory choice. The NFL theorem Dotan’s argument relies upon is a member (...) of a family of theorems in search, optimization, and machine learning. They all claim to show that if no assumptions are made about a search or optimization problem or learning situation, then the best case performance of an algorithm is that of random search or random guessing. A closer inspection shows that these theorems all rely upon assigning uniform probabilities over problems or learning situations, which is just the Principle of Indifference. A counterexample can be crafted that shows that NFL cannot be coherently applied across different descriptions of the same learning situation. To avoid this counterexample, Dotan needs to privilege some description of the learning situation faced by scientists. However, this means that NFL cannot be applied since an important assumption about the problem is being made. So Dotan faces a dilemma: either NFL leads to incoherent best-case partial beliefs or it is inapplicable to the problem of theory choice. This negative result has implications for the larger debate over theory choice. (shrink)

It is undeniable that inductive reasoning has brought us much good. At least since Hume, however, philosophers have wondered how to justify our reliance on induction. In important recent work, Schurz points out that philosophers have been wrongly assuming that justifying induction is tantamount to showing induction to be reliable. According to him, to justify our reliance on induction, it is enough to show that induction is optimal. His optimality approach consists of two steps: an analytic argument for meta-induction (that (...) is, induction over inductive methods) and an application of meta-induction to empirical findings concerning the predictive accuracy of our actual inductive practices as compared to various non-inductive methods. The present article agrees with Schurz’s general conclusion but also argues that it leaves an important aspect of our inductive practices unaccounted for, to wit, that these practices have so far been highly successful; to satisfy the optimality requirement, induction only needs to be as successful as random guessing. To explain the success of induction, I turn to insights from social epistemology, also highlighted in Schurz’s work, and make these formally explicit in a framework that combines the Hegselmann–Krause model for opinion dynamics and Carnap’s system of λ rules. Within this framework, I use a standard evolutionary algorithm to show how communities of interacting agents may, through variation and selection, come to consist of highly successful inductive reasoners. (shrink)

According to the Feigl–Reichenbach–Salmon–Schurz pragmatic justification of induction, no predictive method is guaranteed or even likely to work for predicting the future; but if anything will work, induction will work—at least when induction is employed at the meta-level of predictive methods in light of their track records. One entertains a priori all manner of esoteric prediction methods, and is said to arrive a posteriori at the conclusion, based on the actual past, that object-level induction is optimal. Schurz’s refinements largely solve (...) the notorious short-run problem. A difficulty is noted, however, related to short-run worries but based on localized disagreement about the past, a feature characteristic of real debates (especially early modern) involving induction in intellectual history. Given the evidence about past events, _unfiltered by induction_, meta-induction might support a partly non-inductive method—especially as judged by proponents of esoteric prediction methods, who presumably believe that their methods have worked. Thus induction is justified meta-inductively in contexts where it was uncontroversial, while not obviously justified in key contexts where it has been disputed. This objection, momentarily sensed by Reichenbach regarding clairvoyance, is borne out by the Stoics’ use of meta-induction to justify both science and divination and by ancient Hebrew examples of meta-induction. Schurz’s recently introduced criteria for acceptance of testimony play a crucial role in arriving at object-level induction using meta-induction, but one might question them. Given the need for judgment in accepting testimony, it is unclear that the subjectivity of Howson’s Bayesian answer to Hume’s problem is overcome. (shrink)

White (2015) proposes an a priori justification of the reliability of inductive prediction methods based on his thesis of induction-friendliness. It asserts that there are by far more induction-friendly event sequences than induction-unfriendly event sequences. In this paper I contrast White's thesis with the famous no free lunch (NFL) theorem. I explain two versions of this theorem, the strong NFL theorem applying to binary and the weak NFL theorem applying to real-valued predictions. I show that both versions refute the thesis (...) of induction-friendliness. In the conclusion I argue that an a priori justification of the reliability of induction based on a uniform probability distribution over possible event sequences is impossible. In the outlook I consider two alternative approaches: (i) justification externalism and (ii) optimality justifications. (shrink)

Some scholars claim that epistemology of science and machine learning are actually overlapping disciplines studying induction, respectively affected by Hume's problem of induction and its formal machine-learning counterpart, the “no-free-lunch” (NFL) theorems, to which even advanced AI systems such as LLMs are not immune. Extending Kevin Korb's view, this paper envisions a hierarchy of disciplines where the lowermost is a basic science, and, recursively, the metascience at each level inductively learns which methods work best at the immediately lower level. Due (...) to Hume's dictum and NFL theorems, no exact metanorms for the good performance of each object science can be obtained after just a finite number of levels up the hierarchy, and the progressive abstractness of each metadiscipline and consequent ill-definability of its methods and objects makes science—as defined by a minimal standard of scientificity—cease to exist above a certain metalevel, allowing for a still rational style of inquiry into science that can be called “philosophical.” Philosophical levels, transitively reflecting on science, peculiarly manifest a non–empirically learned urge to self-reflection constituting the properly normative aspect of philosophy of science. (shrink)

One of the most discussed issues in psychology—presently and in the past—is how to define and measure the extent to which human cognition is rational. The rationality of human cognition is often evaluated in terms of normative standards based on a priori intuitions. Yet this approach has been challenged by two recent developments in psychology that we review in this article: ecological rationality and descriptivism. Going beyond these contributions, we consider it a good moment for psychologists and philosophers to join (...) forces and work toward a new foundation for the definition of rational cognition. We take a first step in this direction by proposing that the rationality of both cognitive and normative systems can be measured in terms of their cognitive success. Cognitive success can be defined and gauged in terms of two factors: ecological validity (the system's validity in conditions in which it is applicable) and the system's applicability (the scope of conditions under which it can be applied). As we show, prominent systems of reasoning—deductive reasoning, Bayesian reasoning, uncertain conditionals, and prediction and choice—perform rather differently on these two factors. Furthermore, we demonstrate that conceptualizing rationality according to its cognitive success offers a new perspective on the time-honored relationship between the descriptive (“is”) and the normative (“ought”) in psychology and philosophy. (shrink)