This paper seeks to analyse the effects on Economics of Research Assessment Systems, such as the Research Assessment Exercise (or RAE) which was carried out in the UK between 1986 and 2008. The paper begins by pointing out that, in the 2008 RAE, economics turned out to be the research area which was accorded the highest valuation of any subject in the UK, even though economists were then under attack for failing to predict the global financial crash which had occurred (...) a few months earlier. One aim of the paper is to explain this economics anomaly in research assessment. The paper goes on to point out a key difference between economics and the natural sciences. Most areas of the natural sciences are dominated for most of the time by a single, generally accepted, paradigm, whereas there are always in economics different schools of thought which have different and highly conflicting paradigms. Given this situation, it is argued that the effect of research assessment systems in economics is to strengthen the majority school in the subject (whatever that is), and weaken the minority schools. This conclusion is supported by empirical data collected by Frederic Lee for the UK. It is then shown that the greater the dominance of the majority school, the higher the overall valuation of the subject is likely to be, and this is used to explain the anomaly noted earlier. It is argued that research in economics flourishes better in a situation in which there are a number of different schools treated equally, than in one in which a single school dominates. The conclusion is that research assessment systems have a negative effect on research in economics and give misleading results. Instead of such systems, an attempt should be made to encourage pluralism in the subject. (shrink)

The Research Assessment Exercise was introduced in 1986 by Thatcher, and was continued by Blair. So it has now been running for 21 years. During this time, the rules governing the RAE have changed considerably, and the interval between successive RAEs has also varied. These changes are not of great importance as far as the argument of this paper is concerned. We will concentrate on the main features of the RAE which can be summarised as follows.

Computer simulation of an epistemic landscape model, modified to include explicit representation of a centralized funding body, show the method of funding allocation has significant effects on communal trade-off between exploration and exploitation, with consequences for the community’s ability to generate significant truths. The results show this effect is contextual, and depends on the size of the landscape being explored, with funding that includes explicit random allocation performing significantly better than peer review on large landscapes. The article proposes a way (...) of incorporating external institutional factors in formal social epistemology, and offers a way of bringing such investigations to bear on current research policy questions. 1Introduction2Theoretical Background3Model Description4Simulation Details 4.1Simulating the epistemic landscape4.2Simulating agents4.3Simulating communal knowledge4.4Simulating funding strategies4.5Simulating merit dynamics5Results and Discussion 5.1Experiment 1: The winner-takes-it-all mechanism only5.2Experiment 2: All dynamic mechanisms5.3Experiment 3: Adding a new funding mechanism 5.4Experiment 4: Varying the degree of myopia5.5Experiment 5: Variability of individual epistemic gain5.6Experiment 6: Likelihood of renewal6Discussion7Conclusion. (shrink)

There are now several proposals for introducing random elements into the process of funding allocation for research, and some initial implementation of this policy by funding bodies. The proposals have been supported on efficiency grounds, with models, including social epistemology models, showing random allocation could increase the generation of significant truths in a community of scientists when compared to funding by peer review. The models in the literature are, however, fairly abstract. This paper introduces some of the considerations that are (...) required to build on the modelling work towards a fully-fledged policy proposal, including issues of cost and fairness. (shrink)

Computer simulation of an epistemic landscape model, modified to include explicit representation of a centralized funding body, show the method of funding allocation has significant effects on communal trade-off between exploration and exploitation, with consequences for the community’s ability to generate significant truths. The results show this effect is contextual, and depends on the size of the landscape being explored, with funding that includes explicit random allocation performing significantly better than peer-review on large landscapes. The paper proposes a way of (...) incorporating external institutional factors in formal social epistemology, and offers a way of bringing such investigations to bear on current research policy questions. (shrink)

Because of its complexity, contemporary scientific research is almost always tackled by groups of scientists, each of which works in a different part of a given research domain. We believe that understanding scientific progress thus requires understanding this division of cognitive labor. To this end, we present a novel agent-based model of scientific research in which scientists divide their labor to explore an unknown epistemic landscape. Scientists aim to climb uphill in this landscape, where elevation represents the significance of the (...) results discovered by employing a research approach. We consider three different search strategies scientists can adopt for exploring the landscape. In the first, scientists work alone and do not let the discoveries of the community as a whole influence their actions. This is compared with two social research strategies, which we call the follower and maverick strategies. Followers are biased towards what others have already discovered, and we find that pure populations of these scientists do less well than scientists acting independently. However, pure populations of mavericks, who try to avoid research approaches that have already been taken, vastly outperform both of the other strategies. Finally, we show that in mixed populations, mavericks stimulate followers to greater levels of epistemic production, making polymorphic populations of mavericks and followers ideal in many research domains. (shrink)