The no-free-lunch theorems promote a skeptical conclusion that all possible machine learning algorithms equally lack justification. But how could this leave room for a learning theory, that shows that some algorithms are better than others? Drawing parallels to the philosophy of induction, we point out that the no-free-lunch results presuppose a conception of learning algorithms as purely data-driven. On this conception, every algorithm must have an inherent inductive bias, that wants justification. We argue that many standard learning algorithms should rather (...) be understood as model-dependent: in each application they also require for input a model, representing a bias. Generic algorithms themselves, they can be given a model-relative justification. (shrink)

This paper studies the use of hypotheses schemes in generatinginductive predictions. After discussing Carnap–Hintikka inductive logic,hypotheses schemes are defined and illustrated with two partitions. Onepartition results in the Carnapian continuum of inductive methods, the otherresults in predictions typical for hasty generalization. Following theseexamples I argue that choosing a partition comes down to making inductiveassumptions on patterns in the data, and that by choosing appropriately anyinductive assumption can be made. Further considerations on partitions makeclear that they do not suggest any solution (...) to the problem of induction.Hypotheses schemes provide the tools for making inductive assumptions, but theyalso reveal the need for such assumptions. (shrink)