It is generally accepted among philosophers of science that hypothesis testing is a key methodological feature of science. As far as philosophical theories of confirmation are con...

Preregistration has been proposed as a useful method for making a publicly verifiable distinction between confirmatory hypothesis tests, which involve planned tests of ante hoc hypotheses, and exploratory hypothesis tests, which involve unplanned tests of post hoc hypotheses. This distinction is thought to be important because it has been proposed that confirmatory hypothesis tests provide more compelling results (less uncertain, less tentative, less open to bias) than exploratory hypothesis tests. In this article, we challenge this proposition (...) and argue that there are several advantages of exploratory hypothesis tests that can make their results more compelling than those of confirmatory hypothesis tests. We also consider some potential disadvantages of exploratory hypothesis tests and conclude that their advantages can outweigh the disadvantages. We conclude that exploratory hypothesis tests avoid researcher commitment and researcher prophecy biases, reduce the probability of data fraud, are more appropriate in the context of unplanned deviations, facilitate inference to the best explanation, and allow peer reviewers to make additional contributions at the data analysis stage. In contrast, confirmatory hypothesis tests may lead to an inappropriate level of confidence in research conclusions, less appropriate analyses in the context of unplanned deviations, and greater bias and errors in theoretical inferences. (shrink)

“Dutch Book” arguments and references to gambling theorems are typical in the debate between Bayesians and scientists committed to “classical” statistical methods. These arguments have rarely convinced non-Bayesian scientists to abandon certain conventional practices, partially because many scientists feel that gambling theorems have little relevance to their research activities. In other words, scientists “don’t bet.” This article examines one attempt, by Schervish, Seidenfeld, and Kadane, to progress beyond such apparent stalemates by connecting “Dutch Book”–type mathematical results with principles actually endorsed (...) by practicing experimentalists. (shrink)

According to one theory, the brain is a sophisticated hypothesis tester: perception is Bayesian unconscious inference where the brain actively uses predictions to test, and then refine, models about what the causes of its sensory input might be. The brain’s task is simply continually to minimise prediction error. This theory, which is getting increasingly popular, holds great explanatory promise for a number of central areas of research at the intersection of philosophy and cognitive neuroscience. I show how the (...) theory can help us understand striking phenomena at three cognitive levels: vision, sensory integration, and belief. First, I illustrate central aspects of the theory by showing how it provides a nice explanation of why binocular rivalry occurs. Then I suggest how the theory may explain the role of the unified sense of self in rubber hand and full body illusions driven by visuotactile conflict. Finally, I show how it provides an approach to delusion formation that is consistent with one-deficit accounts of monothematic delusions. (shrink)

Each day people are presented with circumstances that may require speculation. Scientists may ponder questions such as why a star is born or how rainbows are made, psychologists may ask social questions such as why people are prejudiced, and military strategists may imagine what the consequences of their actions might be. Speculations may lead to the generation of putative explanations called hypotheses. But it is by checking if hypotheses accurately reflect the encountered facts that lead to sensible behaviour demonstrating a (...) true understanding. If evidence shows a hypothesis to be false, then people should rationally abandon it, especially if there are negative consequences. The aim of this thesis is to examine how effectively people search for evidence in their hypothesis testing to test whether or not their hypotheses are true or false in competitive games. -/- Research findings from six studies of hypothesis testing behaviour in competitive deductive tasks are explored. Chapter by chapter the thesis tests how everyday people, and master chess players, tackle hypothesis testing in mathematical tasks, such as how to solve sequential number sequence puzzles when thinking about an opponent, or how to solve chess problems in a variety of contexts. The implications of the results are discussed in light of aspects of general cognition: such as reasoning, social hypothesis testing and planning. (shrink)

Complex constraints like conditionals ('If A, then B') and probabilistic constraints ('The probability that A is p') pose problems for Bayesian theories of learning. Since these propositions do not express constraints on outcomes, agents cannot simply conditionalize on the new information. Furthermore, a natural extension of conditionalization, relative information minimization, leads to many counterintuitive predictions, evidenced by the sundowners problem and the Judy Benjamin problem. Building on the notion of a `paradigm shift' and empirical research in psychology and economics, I (...) argue that the model of hypothesis testing can explain how people learn complex, theory-laden propositions like conditionals and probability constraints. Theories are formalized as probability distributions over a set of possible outcomes and theory change is triggered by a constraint which is incompatible with the initial theory. This leads agents to consult a higher order probability function, or a 'prior over priors,' to choose the most likely alternative theory which satisfies the constraint. The hypothesis testing model is applied to three examples: learning a simple probabilistic constraint involving coin bias, the sundowners problem for conditional learning, and the Judy Benjamin problem for learning conditional probability constraints. (shrink)

Although logical consistency is desirable in scientific research, standard statistical hypothesis tests are typically logically inconsistent. To address this issue, previous work introduced agnostic hypothesis tests and proved that they can be logically consistent while retaining statistical optimality properties. This article characterizes the credal modalities in agnostic hypothesis tests and uses the hexagon of oppositions to explain the logical relations between these modalities. Geometric solids that are composed of hexagons of oppositions illustrate the conditions for these modalities (...) to be logically consistent. Prisms composed of hexagons of oppositions show how the credal modalities obtained from two agnostic tests vary according to their threshold values. Nested hexagons of oppositions summarize logical relations between the credal modalities in these tests and prove new relations. (shrink)

Simultaneous hypothesis tests can fail to provide results that meet logical requirements. For example, if A and B are two statements such that A implies B, there exist tests that, based on the same data, reject B but not A. Such outcomes are generally inconvenient to statisticians (who want to communicate the results to practitioners in a simple fashion) and non-statisticians (confused by conflicting pieces of information). Based on this inconvenience, one might want to use tests that satisfy logical (...) requirements. However, Izbicki and Esteves shows that the only tests that are in accordance with three logical requirements (monotonicity, invertibility and consonance) are trivial tests based on point estimation, which generally lack statistical optimality. As a possible solution to this dilemma, this paper adapts the above logical requirements to agnostic tests, in which one can accept, reject or remain agnostic with respect to a given hypothesis. Each of the logical requirements is characterized in terms of a Bayesian decision theoretic perspective. Contrary to the results obtained for regular hypothesis tests, there exist agnostic tests that satisfy all logical requirements and also perform well statistically. In particular, agnostic tests that fulfill all logical requirements are characterized as region estimator-based tests. Examples of such tests are provided. (shrink)

According to the pragmatic hypothesis testing theory, how much evidence we require before we believe something varies depending on the expected costs of falsely believing and disbelieving it. This theory has been used in the self-deception debate to explain our tendencies towards self-deceptive belief formation. This article argues that the application of this theory in the self-deception debate has overlooked the distinction between belief and acceptance, and that the theory in all likelihood models acceptance rather than belief, in which (...) case it is probably not relevant to the explanation of self-deception. It is suggested, however, that doxastic error costs might be relevant to explaining some types of self-deception, though they feature in an evolutionary explanation of it rather than a psychological one. (shrink)

This paper explores the epistemological challenges in using computer simulations for two distinct goals: explanation via hypothesis-testing and prediction. It argues that each goal requires different strategies for justifying inferences drawn from simulation results due to different practical and conceptual constraints. The paper identifies unique and shared strategies researchers employ to increase confidence in their inferences for each goal. For explanation via hypothesis-testing, researchers need to address the underdetermination, interpretability, and attribution challenges. In prediction, the emphasis is on (...) the model's ability to generalize across multiple domains. Shared strategies researchers employ to increase confidence in inferences are empirical corroboration of theoretical assumptions and adequacy of computational operationalizations, and this paper argues that these are necessary for explanation via hypothesis-testing but not for prediction. This paper emphasizes the need for a nuanced approach to the epistemology of computer simulation, given the diverse applications of computer simulation in scientific research. Understanding these differences is crucial for both researchers and philosophers of science, as it helps develop appropriate methodologies and criteria for assessing the trustworthiness of computer simulation. (shrink)

We introduce an expected-value theory of linguistic modality that makes reference to expected utility and a likelihood-based confirmation measure for deontics and epistemics, respectively. The account is a probabilistic semantics for deontics and epistemics, yet it proposes that deontics and epistemics share a common core modal semantics, as in traditional possible-worlds analysis of modality. We argue that this account is not only theoretically advantageous, but also has far-reaching empirical consequences. In particular, we predict modal versions of reasoning fallacies from the (...) heuristics and biases literature. Additionally, we derive the modal semantics in an entirely transparent manner, as it is based on the compositional semantics of Korean modal expressions that are morphosyntactically decomposed into a conditional and an evaluative predicate. (shrink)

This manuscript is about how to start from relying on science to be more certain about Christianity. This is because science is so pervasive in our everyday life that we expect that it works almost every time. However, when it comes to Christianity, we need to have faith because most of the time God does not appear to respond to us. Therefore, we feel uncertain about beliefs in Christianity. Instead of being uncertain, this manuscript tries to find a way so (...) that we are more certain about our beliefs in Christianity. We borrow our method to be certain from science. Science faces a similar situation as in believing in Christianity because its knowledge is fallible. Scientists themselves cannot be certain that the scientific knowledge is true all the time. So, they rely on a method called hypothesis testing to make their decisions to believe or not in scientific knowledge. We borrow this method to believe in Christianity. Therefore, we need evidence to make our decision of belief. We mention some evidence from past miracles to help us to make decisions. These include Eucharistic miracles, Marian apparitions, incorruptible corpses, etc. After we decide to believe in Christianity, we work out a theory of Christianity based on the salvation plan of God. We try to substantiate the principles in the theory by referencing testimonies from the Bible and from evidence of miracles that happened. We show that these principles are related to models and experiments that we formulate based on our beliefs in Christianity. Having obtained an understanding of Christianity like a (historical) science, we formulate our practice as a Christian. (shrink)

The purpose of this paper is to provide a detailed technical protocol analysis of chess masters' evaluative expertise, paying particular attention to the analysis of the structure of their memory process in evaluating foreseen possibilities in games of dynamic equilibrium. The paper has two purposes. First, to publish a results chapter from my DPhil thesis (in revised journal article form) attending to the measurement of foresight in chess masters' evaluation process, testing alternative theories of cognitive expertise in the domain of (...) chess; and second to provide a subset of the technical graphical analysis that corresponds to that measurement to preserve this protocol analysis for access in the academic domain for future studies of expert memory and foresight (e.g., Ericsson & Simon, 1993). The step-by-step protocol analysis consists of: (i) an introduction to foresight cognition as hypothesis testing, (ii) a theoretical review in the domain of chess masters' expertise according to the theoretical frameworks in that field purporting hypotheses relevant to chess masters' evaluative skill processes, and (iii) summary tables and non-parametric statistical analysis corroborating chunking theory frameworks of expert cognition (e.g., DeGroot, 1965; Newell & Simon, 1972; Gobet, 1998; Gobet et al., 2004), and refuting the alternative search-evaluation models (e.g., Holding & Reynolds, 1982). Moreover, the journal article espouses the preservation of the traditional protocol analysis method core to the field of expert cognition (DeGroot, 1969; Kotov, 1971). The full protocol analysis can be found in monograph form here on my SSRN profile in ‘The role of falsification in hypothesis testing’. It takes the form of a specialist population study (e.g., detailed case study work; Luria, 1987). Thus the outline consists of a short introduction, a theoretical methodological review discussing protocol analysis methods for specialist population studies in cognition (with particular attention to the preservation of protocol analysis methods for chess studies in cognition and expert memory/ with a fresh angle on the foresight process), and the full set of protocol analyses with corresponding problem behaviour graphs. A subset of the main results has been published elsewhere (e.g., Cowley & Byrne, 2004; Cowley 2006), receiving scientific and scientific journalistic acclaim (e.g., Nature Online News 2004). (shrink)

Falsification may demarcate science from non-science as the rational way to test the truth of hypotheses. But experimental evidence from studies of reasoning shows that people often find falsification difficult. We suggest that domain expertise may facilitate falsification. We consider new experimental data about chess experts’ hypothesis testing. The results show that chess masters were readily able to falsify their plans. They generated move sequences that falsified their plans more readily than novice players, who tended to confirm their (...) plans. The finding that experts in a domain are more likely to falsify their hypotheses has important implications for the debate about human rationality. (shrink)

Fodor argued that learning a concept by hypothesis testing would involve an impossible circularity. I show that Fodor's argument implicitly relies on the assumption that actually φ-ing entails an ability to φ. But this assumption is false in cases of φ-ing by luck, and just such luck is involved in testing hypotheses with the kinds of generative random sampling methods that many cognitive scientists take our minds to use. Concepts thus can be learned by hypothesis testing without circularity, (...) and it is plausible that this is how humans in fact acquire at least some of their concepts. (shrink)

In this article I give a critical evaluation of the use and limitations of null-model-based hypothesis testing as a research strategy in the biological sciences. According to this strategy, the null model based on a randomization procedure provides an appropriate null hypothesis stating that the existence of a pattern is the result of random processes or can be expected by chance alone, and proponents of other hypotheses should first try to reject this null hypothesis in order to (...) demonstrate their own hypotheses. Using as an example the controversy over the use of null hypotheses and null models in species co-occurrence studies, I argue that null-model-based hypothesis testing fails to work as a proper analog to traditional statistical null-hypothesis testing as used in well-controlled experimental research, and that the random process hypothesis should not be privileged as a null hypothesis. Instead, the possible use of the null model resides in its role of providing a way to challenge scientists’ commonsense judgments about how a seemingly unusual pattern could have come to be. Despite this possible use, null-model-based hypothesis testing still carries certain limitations, and it should not be regarded as an obligation for biologists who are interested in explaining patterns in nature to first conduct such a test before pursuing their own hypotheses. (shrink)

Two main cognitive theories predict that people find refuting evidence that falsifies their theorising difficult, if not impossible to consider, even though such reasoning may be pivotal to grounding their everyday thoughts in reality (i.e., Poletiek, 1996; Klayman & Ha, 1987). In the classic 2-4-6 number sequence task devised by psychologists to test such reasoning skills in a simulated environment – people fail the test more often than not. In the 2-4-6 task participants try to discover what rule (...) the number triple 2-4-6 conforms to. The rule is ‘ascending numbers’, but it is tricky to discover this rule. Participants tend to generate hypotheses with the properties of the 2-4-6 triple, for example, ‘even numbers ascending in twos’. They must search for evidence to test whether their hypothesis is the rule. But experimental evidence has shown that they tend to generate confirming triples that they expect to be consistent with their hypothesis rather than inconsistent falsifying triples. Counter to the two main hypothesis testing theories this paper demonstrates that falsification is possible in five 2-4-6 task experiments when participants consider an Imaginary Participant’s hypothesis. Experiment 1 and 2 show that competition with an opponent hypothesis tester facilitates falsification. Experiments 3 to 5 show that the consideration of an alternative hypothesis helps this falsification of hypotheses lead to rule discovery. The implications of the results for theories of hypothesis testing and reasoning are discussed. (shrink)

Bostrom's Simulation Argument has become synonymous with simulation theory, despite the existence of alternative frameworks that also suggest our universe may be a computer simulation—frameworks that do not require the assumption of posthumans. Further, Bostrom's work has spurred discussions around the potential destruction or termination of our universe, prompting existential anxieties and fueling extensive philosophical speculation. We evaluate the arguments of Greene, Turchin, Batin, Denkenberger, and Yampolskiy, alongside the counterarguments of Braddon-Mitchell, Latham, and Edge. We conclude that Bostrom's Simulation Argument (...) has been philosophically extended far beyond its original scope. As a contrasting model, we present Campbell's consciousness-based My Big TOE (MBT) Theory. MBT requires fewer assumptions, excludes posthuman entities, reduces the likelihood of simulation termination to near zero, includes a clear cosmogony and telos, and frames existence as an information-entropy-reduction system designed to evolve the consciousnesses of its participants—rather than the existential risk research project framework implicitly assumed in Bostrom's model. (shrink)

Conditional independence tests have received special attention lately in machine learning and computational intelligence related literature as an important indicator of the relationship among the variables used by their models. In the field of probabilistic graphical models, which includes Bayesian network models, conditional independence tests are especially important for the task of learning the probabilistic graphical model structure from data. In this paper, we propose the full Bayesian significance test for tests of conditional independence for discrete datasets. The full (...) Bayesian significance test is a powerful Bayesian test for precise hypothesis, as an alternative to the frequentist’s significance tests (characterized by the calculation of the p-value). (shrink)

In this paper, the notion of degree of inconsistency is introduced as a tool to evaluate the sensitivity of the Full Bayesian Significance Test (FBST) value of evidence with respect to changes in the prior or reference density. For that, both the definition of the FBST, a possibilistic approach to hypothesis testing based on Bayesian probability procedures, and the use of bilattice structures, as introduced by Ginsberg and Fitting, in paraconsistent logics, are reviewed. The computational and theoretical advantages (...) of using the proposed degree of inconsistency based sensitivity evaluation as an alternative to traditional statistical power analysis is also discussed. (shrink)

Since Benjamin Libet’s famous experiments in 1979, the study of the will has become a focal point in the cognitive sciences. Just like Libet the scien-tists Daniel Wegner and Thalia Wheatley came to doubt that the will is causally efficacious. In their influential study I Spy from 1999, they created an experi-mental setup to show that agents erroneously experience their actions as caused by their thoughts. Instead, these actions are caused by unconscious neural pro-cesses; the agent’s ‘causal experience of will’ (...) is just an illusion. Both the scien-tific method and the conclusion drawn from the empirical results have already been criticized by philosophers. In this paper, I will analyze the action performed in I Spy and criticize more fundamentally the assumption of a ‘causal experience of will’. I will argue that the I Spy study does not show that the agent’s causal experience of will is illusory, because it does not show that there is a causal ex-perience of will. Against Wegner and Wheatley’s assumption, I will show that it is unlikely that the participants in I Spy experienced their conscious thoughts as causally efficacious for an action, that they did not perform at all. It is more likely, that they experienced their own bodily movement as causally efficacious for a cooperative action, that they did not perform solely by themselves. (shrink)

-/- For all that digital methods—including network visualization, text analysis, and others—have begun to show extensive promise in philosophical contexts, a tension remains between two uses of those tools that have often been taken to be incompatible, or at least to engage in a kind of trade-off: the discovery of new hypotheses and the testing of already-formulated positions. This paper presents this basic distinction, then explores ways to resolve this tension with the help of two interdisciplinary case studies, taken from (...) preregistration in contemporary science and the debate over whig history in the history of science. These case studies, the paper argues, refocus our attention from a mutually exclusive testing/discovery binary to the relationship between our background data or philosophical views and the empirical generalizations that we might draw from the data. Finally, it develops a set of three challenges for philosophers and corresponding avenues for future work that will, it is hoped, allow us to better justify our use of these methods. (shrink)

The unit root problem plays a central role in empirical applications in the time series econometric literature. However, significance tests developed under the frequentist tradition present various conceptual problems that jeopardize the power of these tests, especially for small samples. Bayesian alternatives, although having interesting interpretations and being precisely defined, experience problems due to the fact that that the hypothesis of interest in this case is sharp or precise. The Bayesian significance test used in this article, for the (...) unit root hypothesis, is based solely on the posterior density function, without the need of imposing positive probabilities to sets of zero Lebesgue measure. Furthermore, it is conducted under strict observance of the likelihood principle. It was designed mainly for testing sharp null hypotheses and it is called FBST for Full Bayesian Significance Test. (shrink)

Gignac and Zajenkowski (2020) find that “the degree to which people mispredicted their objectively measured intelligence was equal across the whole spectrum of objectively measured intelligence”. This Comment shows that Gignac and Zajenkowski’s (2020) finding of homoscedasticity is likely the result of a recoding choice by the experimenters and does not in fact indicate that the Dunning-Kruger Effect is a mere statistical artifact. Specifically, Gignac and Zajenkowski (2020) recoded test subjects’ responses to a question regarding self-assessed comparative IQ onto (...) a linear IQ scale when a normal IQ scale would likely have been more appropriate. More generally, researchers studying self-assessed intelligence should be aware of potential measurement problems that may arise when transforming an ordinal scale onto an interval scale. (shrink)

A new Evidence Test is applied to the problem of testing whether two Poisson random variables are dependent. The dependence structure is that of Holgate’s bivariate distribution. These bivariate distribution depends on three parameters, 0 < theta_1, theta_2 < infty, and 0 < theta_3 < min(theta_1, theta_2). The Evidence Test was originally developed as a Bayesian test, but in the present paper it is compared to the best known test of the hypothesis of independence in (...) a frequentist framework. It is shown that the Evidence Test is considerably more powerful when the correlation is not too close to zero, even for small samples. (shrink)

Gene clustering is a useful exploratory technique to group together genes with similar expression levels under distinct cell cycle phases or distinct conditions. It helps the biologist to identify potentially meaningful relationships between genes. In this study, we propose a clustering method based on multivariate normal mixture models, where the number of clusters is predicted via sequential hypothesis tests: at each step, the method considers a mixture model of m components (m = 2 in the first step) and tests (...) if in fact it should be m - 1. If the hypothesis is rejected, m is increased and a new test is carried out. The method continues (increasing m) until the hypothesis is accepted. The theoretical core of the method is the full Bayesian significance test, an intuitive Bayesian approach, which needs no model complexity penalization nor positive probabilities for sharp hypotheses. Numerical experiments were based on a cDNA microarray dataset consisting of expression levels of 205 genes belonging to four functional categories, for 10 distinct strains of Saccharomyces cerevisiae. To analyze the method’s sensitivity to data dimension, we performed principal components analysis on the original dataset and predicted the number of classes using 2 to 10 principal components. Compared to Mclust (model-based clustering), our method shows more consistent results. (shrink)

Epistemic contextualism is one of the most hotly debated topics in contemporary epistemology. Contextualists claim that ‘know’ is a context-sensitive verb associated with different evidential standards in different contexts. Contextualists motivate their view based on a set of behavioural claims. In this paper, I show that several of these behavioural claims are false. I also show that contextualist test cases suffer from a critical confound, which derives from people's tendency to defer to speakers’ statements about their own mental states. (...) My evidence consists in results from several behavioural experiments. I conclude that contextualism is an idle hypothesis and I propose some general methodological lessons. (shrink)

During multiple testing, researchers often adjust their alpha level to control the familywise error rate for a statistical inference about a joint union alternative hypothesis (e.g., “H1,1 or H1,2”). However, in some cases, they do not make this inference. Instead, they make separate inferences about each of the individual hypotheses that comprise the joint hypothesis (e.g., H1,1 and H1,2). For example, a researcher might use a Bonferroni correction to adjust their alpha level from the conventional level of 0.050 (...) to 0.025 when testing H1,1 and H1,2, find a significant result for H1,1 (p < 0.025) and not for H1,2 (p > .0.025), and so claim support for H1,1 and not for H1,2. However, these separate individual inferences do not require an alpha adjustment. Only a statistical inference about the union alternative hypothesis “H1,1 or H1,2” requires an alpha adjustment because it is based on “at least one” significant result among the two tests, and so it refers to the familywise error rate. Hence, an inconsistent correction occurs when a researcher corrects their alpha level during multiple testing but does not make an inference about a union alternative hypothesis. In the present article, I discuss this inconsistent correction problem, including its reduction in statistical power for tests of individual hypotheses and its potential causes vis-à-vis error rate confusions and the alpha adjustment ritual. I also provide three illustrations of inconsistent corrections from recent psychology studies. I conclude that inconsistent corrections represent a symptom of statisticism, and I call for a more nuanced inference-based approach to multiple testing corrections. (shrink)

Can the theory that reality is a simulation be tested? We investigate this question based on the assumption that if the system performing the simulation is nite (i.e. has limited resources), then to achieve low computational complexity, such a system would, as in a video game, render content (reality) only at the moment that information becomes available for observation by a player and not at the moment of detection by a machine (that would be part of the simulation and whose (...) detection would also be part of the internal computation performed by the Virtual Reality server before rendering content to the player). Guided by this principle we describe conceptual wave/particle duality experiments aimed at testing the simulation theory. (shrink)

To estimate causal relationships, time series econometricians must be aware of spurious correlation, a problem first mentioned by Yule (1926). To deal with this problem, one can work either with differenced series or multivariate models: VAR (VEC or VECM) models. These models usually include at least one cointegration relation. Although the Bayesian literature on VAR/VEC is quite advanced, Bauwens et al. (1999) highlighted that “the topic of selecting the cointegrating rank has not yet given very useful and convincing results”. The (...) present article applies the Full Bayesian Significance Test (FBST), especially designed to deal with sharp hypotheses, to cointegration rank selection tests in VECM time series models. It shows the FBST implementation using both simulated and available (in the literature) data sets. As illustration, standard non informative priors are used. (shrink)

We review the definition of the Full Bayesian Significance Test (FBST), and summarize its main statistical and epistemological characteristics. We review also the Abstract Belief Calculus (ABC) of Darwiche and Ginsberg, and use it to analyze the FBST’s value of evidence. This analysis helps us understand the FBST properties and interpretation. The definition of value of evidence against a sharp hypothesis, in the FBST setup, was motivated by applications of Bayesian statistical reasoning to legal matters where the sharp (...) hypotheses were defendants statements, to be judged according to the Onus Probandi juridical principle. (shrink)

In his [1937, 1938], Paul Dirac proposed his “Large Number Hypothesis” (LNH), as a speculative law, based upon what we will call the “Large Number Coincidences” (LNC’s), which are essentially “coincidences” in the ratios of about six large dimensionless numbers in physics. Dirac’s LNH postulates that these numerical coincidences reflect a deeper set of law-like relations, pointing to a revolutionary theory of cosmology. This led to substantial work, including the development of Dirac’s later [1969/74] cosmology, and other alternative cosmologies, (...) such as the Brans-Dicke modification of GTR, and to extensive empirical tests. We may refer to the generic hypothesis of “Large Number Relations” (LNR’s), as the proposal that there are lawlike relations of some kind between the dimensionless numbers, not necessarily those proposed in Dirac’s early LNH. Such relations would have a profound effect on our concepts of physics, but they remain shrouded in mystery. Although Dirac’s specific proposals for LNR theories have been largely rejected, the subject retains interest, especially among cosmologists seeking to test possible variations in fundamental constants, and to explain dark energy or the cosmological constant. In the first two sections here we briefly summarize the basic concepts of LNR’s. We then introduce an alternative LNR theory, using a systematic formalism to express variable transformations between conventional measurement variables and the true variables of the theory. We demonstrate some consistency results and review the evidence for changes in the gravitational constant G. The theory adopted in the strongest tests of Ġ/G, by the Lunar Laser Ranging (LLR) experiments, assumes: Ġ/G = 3(dr/dt)/r – 2(dP/dt)/P – (dm/dt)/m, as a fundamental relationship. Experimental measurements show the RHS to be close to zero, so it is inferred that significant changes in G are ruled out. However when the relation is derived in our alternative theory it gives: Ġ/G = 3(dr/dt)/r – 2(dP/dt)/P – (dm/dt)/m – (dR/dt)/R. The extra final term (which is the Hubble constant) is not taken into account in conventional derivations. This means the LLR experiments are consistent with our LNR theory (and others), and they do not really test for a changing value of G at all. This failure to transform predictions of LNR theories correctly is a serious conceptual flaw in current experiment and theory. (shrink)

The full Bayesian signi/cance test (FBST) for precise hypotheses is presented, with some illustrative applications. In the FBST we compute the evidence against the precise hypothesis. We discuss some of the theoretical properties of the FBST, and provide an invariant formulation for coordinate transformations, provided a reference density has been established. This evidence is the probability of the highest relative surprise set, “tangential” to the sub-manifold (of the parameter space) that defines the null hypothesis.

This paper introduces pragmatic hypotheses and relates this concept to the spiral of scientific evolution. Previous works determined a characterization of logically consistent statistical hypothesis tests and showed that the modal operators obtained from this test can be represented in the hexagon of oppositions. However, despite the importance of precise hypothesis in science, they cannot be accepted by logically consistent tests. Here, we show that this dilemma can be overcome by the use of pragmatic versions of precise (...) hypotheses. These pragmatic versions allow a level of imprecision in the hypothesis that is small relative to other experimental conditions. The introduction of pragmatic hypotheses allows the evolution of scientific theories based on statistical hypothesis testing to be interpreted using the narratological structure of hexagonal spirals, as defined by Pierre Gallais. (shrink)

The Full Bayesian Significance Test (FBST) for precise hypotheses is presented, with some applications relevant to reliability theory. The FBST is an alternative to significance tests or, equivalently, to p-ualue.s. In the FBST we compute the evidence of the precise hypothesis. This evidence is the probability of the complement of a credible set "tangent" to the sub-manifold (of the para,rreter space) that defines the null hypothesis. We use the FBST in an application requiring a quality control of (...) used components, based on remaining life statistics. (shrink)

The evolution of the myxoma virus in Australia has been presented for many years as a test case for the hypothesis that group selection can function effectively `in the wild.' This paper critically examines the myxoma case, and argues that its failure as a test case for this hypothesis has broader implications for debates over the levels of selection.

Much forensic inference based upon DNA evidence is made assuming Hardy-Weinberg Equilibrium (HWE) for the genetic loci being used. Several statistical tests to detect and measure deviation from HWE have been devised, and their limitations become more obvious when testing for deviation within multiallelic DNA loci. The most popular methods-Chi-square and Likelihood-ratio tests-are based on asymptotic results and cannot guarantee a good performance in the presence of low frequency genotypes. Since the parameter space dimension increases at a quadratic rate on (...) the number of alleles, some authors suggest applying sequential methods, where the multiallelic case is reformulated as a sequence of “biallelic” tests. However, in this approach it is not obvious how to assess the general evidence of the original hypothesis; nor is it clear how to establish the significance level for its acceptance/rejection. In this work, we introduce a straightforward method for the multiallelic HWE test, which overcomes the aforementioned issues of sequential methods. The core theory for the proposed method is given by the Full Bayesian Significance Test (FBST), an intuitive Bayesian approach which does not assign positive probabilities to zero measure sets when testing sharp hypotheses. We compare FBST performance to Chi-square, Likelihood-ratio and Markov chain tests, in three numerical experiments. The results suggest that FBST is a robust and high performance method for the HWE test, even in the presence of several alleles and small sample sizes. (shrink)

In this research, we focus on the usage of adversarial sampling to test for the fairness in the prediction of deep neural network model across different classes of image in a given dataset. While several framework had been proposed to ensure robustness of machine learning model against adversarial attack, some of which includes adversarial training algorithm. There is still the pitfall that adversarial training algorithm tends to cause disparity in accuracy and robustness among different group. Our research is aimed (...) at using adversarial sampling to test for fairness in the prediction of deep neural network model across different classes or categories of image in a given dataset. We successfully demonstrated a new method of ensuring fairness across various group of input in deep neural network classifier. We trained our neural network model on the original image, and without training our model on the perturbed or attacked image. When we feed the adversarial samplings to our model, it was able to predict the original category/ class of the image the adversarial sample belongs to. We also introduced and used the separation of concern concept from software engineering whereby there is an additional standalone filter layer that filters perturbed image by heavily removing the noise or attack before automatically passing it to the network for classification, we were able to have accuracy of 93.3%. Cifar-10 dataset have ten categories of dataset, and so, in order to account for fairness, we applied our hypothesis across each categories of dataset and were able to get a consistent result and accuracy. (shrink)

Statistical tests that detect and measure deviation from the Hardy-Weinberg equilibrium (HWE) have been devised but are limited when testing for deviation at multiallelic DNA loci is attempted. Here we present the full Bayesian significance test (FBST) for the HWE. This test depends neither on asymptotic results nor on the number of possible alleles for the particular locus being evaluated. The FBST is based on the computation of an evidence index in favor of the HWE hypothesis. A (...) great deal of forensic inference based on DNA evidence assumes that the HWE is valid for the genetic loci being used. We applied the FBST to genotypes obtained at several multiallelic short tandem repeat loci during routine parentage testing; the locus Penta E exemplifies those clearly in HWE while others such as D10S1214 and D19S253 do not appear to show this. (shrink)

It has recently been argued that a person’s moral judgments (about both their own and others’ actions) are constrained by the nature and extent of their relevant ignorance and, thus, that such judgments are determined in the first instance by the person’s epistemic circumstances. It has been argued, in other words, that the epistemic is logically prior to other normative (e.g., ethical, prudential, pecuniary) considerations in human decision-making, that these other normative considerations figure in decision-making only after (logically and temporally) (...) relevant ignorance has constrained the decision-maker’s menu of options. If this is right, then a person’s moral judgments in some set of circumstances should vary with their knowledge and ignorance of these circumstances. In this study, we test the hypothesis of the logical priority of the epistemic. We describe two experiments in which subjects’ knowledge and ignorance of relevant consequences were manipulated. In the second experiment, we also compared the effect of ignorance on moral judgments with that of personal force, a factor previously shown to influence moral judgments. We found broad empirical support for the armchair arguments that epistemic considerations are logically prior to normative considerations. (shrink)

Abstract: The Pull Bayesian Significance Test (FBST) for precise hy- potheses is applied to a Multivariate Normal Structure (MNS) model. In the FBST we compute the evidence against the precise hypothesis. This evi- dence is the probability of the Highest Relative Surprise Set (HRSS) tangent to the sub-manifold (of the parameter space) that defines the null hypothesis. The MNS model we present appears when testing equivalence conditions for genetic expression measurements, using micro-array technology.

Scientists often adjust their significance threshold during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that (...) alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses. (shrink)

The Ramsey Test is considered to be the default test for the acceptability of indicative conditionals. I will argue that it is incompatible with some of the recent developments in conceptualizing conditionals, namely the growing empirical evidence for the _Relevance Hypothesis_. According to the hypothesis, one of the necessary conditions of acceptability for an indicative conditional is its antecedent being positively probabilistically relevant for the consequent. The source of the idea is _Evidential Support Theory_ presented in Douven (...) (2008). I will defend the hypothesis against alleged counterexamples, and show that it is supported by growing empirical evidence. Finally, I will present a version of the Ramsey test which incorporates the relevance condition and therefore is consistent with growing empirical evidence for the relevance hypothesis. (shrink)

Pierre Duhem’s influential argument for holism relies on a view of the role that background theory plays in testing: according to this still common account of “auxiliary hypotheses,” elements of background theory serve as truth-apt premises in arguments for or against a hypothesis. I argue that this view is mistaken. Rather than serving as truth-apt premises in arguments, auxiliary hypotheses are employed as “epistemic tools”: instruments that perform specific tasks in connecting our theoretical questions with the world but that (...) are not premises in arguments. On the resulting picture, the acceptability of an auxiliary hypothesis depends not on its truth but on contextual factors such as the task or purpose it is put to and the other tools employed alongside it. (shrink)

An impressive case has been built for the hypothesis that knowledge is the norm of assertion, otherwise known as the knowledge account of assertion. According to the knowledge account, you should assert something only if you know that it’s true. A wealth of observational data supports the knowledge account, and some recent empirical results lend further, indirect support. But the knowledge account has not yet been tested directly. This paper fills that gap by reporting the results of such a (...) test. The knowledge account passes with flying colors. (shrink)

There are many social psychological theories regarding the nature of prejudice, but only one major theory of prejudice reduction: under the right circumstances, prejudice between groups will be reduced with increased contact. On the one hand, the contact hypothesis has a range of empirical support and has been a major force in social change. On the other hand, there are practical and ethical obstacles to any large-scale controlled test of the hypothesis in which relevant variables can be (...) manipulated. Here we construct a spatialized model that tests the core hypothesis in a large array of game-theoretic agents. Robust results offer a new kind of support for the contact hypothesis: results in simulation do accord with a hypothesis of reduced prejudice with increased contact. The spatialized game-theoretic model also suggests a deeper explanation for at least some of the social psychological phenomena at issue. (shrink)

It has recently been argued that a person’s moral judgments (about both their own and others’ actions) are constrained by the nature and extent of their relevant ignorance and, thus, that such judgments are determined in the first instance by the person’s epistemic circumstances. It has been argued, in other words, that the epistemic is logically prior to other normative (e.g., ethical, prudential, pecuniary) considerations in human decision-making, that these other normative considerations figure in decision-making only after (logically and temporally) (...) relevant ignorance has constrained the decision-maker’s menu of options. If this is right, then a person’s moral judgments in some set of circumstances should vary with their knowledge and ignorance of these circumstances. In this study, we test the hypothesis of the logical priority of the epistemic. We describe two experiments in which subjects’ knowledge and ignorance of relevant consequences were manipulated. In the second experiment, we also compared the effect of ignorance on moral judgments with that of personal force, a factor previously shown to influence moral judgments. We found broad empirical support for the armchair arguments that epistemic considerations are logically prior to normative considerations. (shrink)

The “received wisdom” in contemporary analytic philosophy is that intuition talk is a fairly recent phenomenon, dating back to the 1960s. In this paper, we set out to test two interpretations of this “received wisdom.” The first is that intuition talk is just talk, without any methodological significance. The second is that intuition talk is methodologically significant; it shows that analytic philosophers appeal to intuition. We present empirical and contextual evidence, systematically mined from the JSTOR corpus and HathiTrust’s Digital (...) Library, which provide some empirical support for the second rather than the first hypothesis. Our data also suggest that appealing to intuition is a much older philosophical methodology than the “received wisdom” alleges. We then discuss the implications of our findings for the contemporary debate over philosophical methodology. (shrink)

1: Placebo effect sometimes plausibly has powerful effects. 2: Buddhism philosophy holds that every living animal is sentient, and has a consciousness and killing them is bad for the killer and freeing them is good somehow. Consciousness may involve some aspects of quantum mechanism as Roger Penrose’s Orch OR' theory of consciousness pointed out. -/- 3: Although previous studies show that there is no “demonstrable effects of distant intercessory prayer. Anecdotal cases of a special kind of Buddhist activity of freeing (...) captive animals in huge amount seem to have powerful therapeutic effects on severe and terminal patients. We want test it experimentally. 4: The cunning point that exceeds previous stuies is, it’s reasonable to consider there may be collective and superpositional effects of this activity due to the likely quantum mechanism of consciousness, and it may play as a sort of magnifying glass that shows a discernible placebo effect. -/- 5:We recruit a considerable number of patients that are terminally ill and the doctors said they can not help them anymore and have double-blind experiments of the above idea.The statistics may show some anomalous distribution. Ethical concerns shall take into consideration. (shrink)

Eyal Shahar’s essay review [1] of James Penston’s remarkable book [2] seems more inspired playful academic provocation than review or essay, expressing dramatic views of impossible validity. The account given of modern biostatistical causation reveals the slide from science into the intellectual confusion and non-science RCTs have created: “…. the purpose of medical research is to estimate the magnitude of the effect of a causal contrast, for example the probability ratio of a binary outcome …” But Shahar’s world is simultaneously (...) not probabilistic, but of absolute uncertainty: “We should have no confidence in any type of evidence ….. We should have no confidence at all”. Shahar’s "Causal contrast" is attractive. It seems to make sense, but bypasses in two words the means of establishing causation by the scientific method. This phrase assumes a numeric statistically significant “contrast” is causal rather than a potential correlation requiring further investigation. The concept of “causal contrast” is a slippery slope from sense into biostatistical non-science. This can be illustrated with an hypothetical RCT where 100% of interventions exhibit a posited treatment effect and 0% of placebo controls. Internal validity is seemingly quite reasonably assumed satisfied (common-sense dictating the likelihood of an awesome magnificent fraud, bias or plain error of the magnitude required is infinitesimal). Scientific method appears satisfied. The RCT demonstrates: (1) strict regularity of outcome in the presence of posited cause; (2) the absence of outcome in its absence and (3) an intervention (experiment) showing the direction of causation is from posited cause to posited effect. Now travel further down the slope from science. Assume 50% of interventions and 0% of controls are positive. We compromise scientific method, but justify this by assuming a large subgroup which we say surely must on these figures be exhibiting the posited treatment effect. But what of 10% of interventions and 9% of placebo controls exhibiting the posited treatment effect? Our biostatistician says the 1% “causal contrast” is statistically significant. But we have: (1) minimal evidence of regularity; (2) the posited outcome irrespective of presence of posited cause and (3) our intervention is at the highest equivocal in demonstrating any form of causation. This is not science. It is, however, where biostatistics has unthinkingly taken us, as Penston has shown comprehensively [2]. We, the audience of published medical research, are now for the 10% / 9% example well down the slope from science. An unattractive hypothesis results requiring numerous assumptions similar to these:- "There is a 'contrast' which is ‘causal’, albeit the method employed is not scientific. An effect of the intervention has been observed in a very small subgroup. This subgroup is susceptible to treatment. The similar number of placebo controls exhibiting the outcome sought is irrelevant, because the 1% difference between intervention and controls is statistically significant. The statistical analysis is valid and reliable. The RCT’s internal validity is sufficiently satisfied. No funding or bias or fraud has affected the results or their analysis.” As Penston notes: “Confirming and refuting the results of research is crucial to science …. But … there’s no way of testing the results of any particular large-scale RCT or epidemiological study. Each study … is left hanging in the air, unsupported.” It gets worse. To identify a rare serious adverse reaction of a frequency of 1:10,000 can require a trial of 200,000 or larger split between controls and interventions. This is not done. But for every 100 who prospectively benefit from the intervention, 9,900 also receive it. And for every 100 benefiting one person (who likely gains no benefit) will suffer a serious unidentified adverse reaction. This is also without taking account of more common adverse reactions whether serious or otherwise. References [1] Shahar, E. (2011) Research and medicine: human conjectures at every turn. International Journal of Person Centered Medicine 1 (2), 250-253. [2] Penston, J. (2010). stats.con: How we’ve been fooled by statistics-based research in medicine. London: The London Press, UK. (shrink)