Machine learning operates at the intersection of statistics and computer science. This raises the question as to its underlying methodology. While much emphasis has been put on the close link between the process of learning from data and induction, the falsificationist component of machine learning has received minor attention. In this paper, we argue that the idea of falsification is central to the methodology of machine learning. It is commonly thought that machine learning algorithms infer general prediction rules from past (...) observations. This is akin to a statistical procedure by which estimates are obtained from a sample of data. But machine learning algorithms can also be described as choosing one prediction rule from an entire class of functions. In particular, the algorithm that determines the weights of an artificial neural network operates by empirical risk minimization and rejects prediction rules that lack empirical adequacy. It also exhibits a behavior of implicit regularization that pushes hypothesis choice toward simpler prediction rules. We argue that taking both aspects together gives rise to a falsificationist account of artificial neural networks. (shrink)

The idea that “simplicity is a sign of truth”, and the related “Occam’s razor” principle, stating that, all other things being equal, simpler models should be preferred to more complex ones, have been long discussed in philosophy and science. We explore these ideas in the context of supervised machine learning, namely the branch of artificial intelligence that studies algorithms which balance simplicity and accuracy in order to effectively learn about the features of the underlying domain. Focusing on statistical learning theory, (...) we show that situations exist for which a preference for simpler models provably slows down, instead of favoring, the supervised learning process. Our results shed new light on the relations between simplicity and truth approximation, which are briefly discussed in the context of both machine learning and the philosophy of science. (shrink)

Some scholars claim that epistemology of science and machine learning are actually overlapping disciplines studying induction, respectively affected by Hume's problem of induction and its formal machine-learning counterpart, the “no-free-lunch” (NFL) theorems, to which even advanced AI systems such as LLMs are not immune. Extending Kevin Korb's view, this paper envisions a hierarchy of disciplines where the lowermost is a basic science, and, recursively, the metascience at each level inductively learns which methods work best at the immediately lower level. Due (...) to Hume's dictum and NFL theorems, no exact metanorms for the good performance of each object science can be obtained after just a finite number of levels up the hierarchy, and the progressive abstractness of each metadiscipline and consequent ill-definability of its methods and objects makes science—as defined by a minimal standard of scientificity—cease to exist above a certain metalevel, allowing for a still rational style of inquiry into science that can be called “philosophical.” Philosophical levels, transitively reflecting on science, peculiarly manifest a non–empirically learned urge to self-reflection constituting the properly normative aspect of philosophy of science. (shrink)