Methodology is often a point of contention in gender-based salary studies. Although this debate seems at first to be merely about technical issues, it also has an important conceptual dimension. We argue that there are two competing implicit conceptions of discrimination, one institutional and the other individual, that underlie many such debates. We first contrast the preferred methodologies advanced by each side, the policy capturing approach and the flagging approach, and explore the theoretical meaning of their statistical models. We then (...) describe a practical application of both methodological approaches in one specific salary inequity study. In conclusion, we reflect on the implications of such practical statistical choices, discuss how such models can be combined, and make suggestions for sociologists who act as statistical experts or work with them in gender-based salary inequity studies on their own campuses. (shrink)

There is an ongoing and robust tradition of science and technology studies scholars conducting ethnographic laboratory studies. These laboratory studies—like all ethnographies—are each conducted at a particular time, are situated in a particular place, and are about a particular culture. Presumably, this contextual specificity means that such ethnographies have limited applicability beyond the narrow slice of time, place, and culture that they each subject to examination. But we do not always or even often treat them that way. It is beyond (...) common for us to speak about what one or another laboratory study reveals about the laboratory, or “science” much more broadly. Given the contextual specificity of our ethnographic laboratory studies, what justifies this presumed generalizability? Initially, this manuscript surveys typical responses to this question, but then it pursues an unusual one: the potential replicability of ethnographic results. This potential is hereby explored, via an ethnographic replication attempt—one designed and conducted in order to test the generalizability of a particular laboratory study, that of Latour and Woolgar’s classic Laboratory Life. The results of the ethnographic replication attempt are reported, and a remarkable degree of replicability is established. (shrink)

This study develops a Science–Technology–Society (STS)-based science ethics education program for high school students majoring in or planning to major in science and engineering. Our education program includes the fields of philosophy, history, sociology and ethics of science and technology, and other STS-related theories. We expected our STS-based science ethics education program to promote students’ epistemological beliefs and moral judgment development. These psychological constructs are needed to properly solve complicated moral and social dilemmas in the fields of science and engineering. (...) We applied this program to a group of Korean high school science students gifted in science and engineering. To measure the effects of this program, we used an essay-based qualitative measurement. The results indicate that there was significant development in both epistemological beliefs and moral judgment. In closing, we briefly discuss the need to develop epistemological beliefs and moral judgment using an STS-based science ethics education program. (shrink)

Current debates about the replication crisis in psychology take it for granted that direct replication is valuable and focus their attention on questionable research practices in regard to statistical analyses. This paper takes a broader look at the notion of replication as such. It is argued that all experimentation/replication involves individuation judgments and that research in experimental psychology frequently turns on probing the adequacy of such judgments. In this vein, I highlight the ubiquity of conceptual and material questions in research, (...) and I argue that replication is not as central to psychological research as it is sometimes taken to be. (shrink)

The use of neuroscientific evidence in criminal trials has been steadily increasing. Despite progress made in recent decades in understanding the mechanisms of psychological and behavioral functioning, neuroscience is still in an early stage of development and its potential for influencing legal decision-making is highly contentious. Scholars disagree about whether or how neuroscientific evidence might impact prescriptions of criminal culpability, particularly in instances in which evidence of an accused’s history of mental illness or brain abnormality is offered to support a (...) plea of not criminally responsible. In the context of these debates, philosophers and legal scholars have identified numerous problems with admitting neuroscientific evidence in legal contexts. To date, however, less has been said about the challenges of evaluating the evidence upon which integrative mechanistic explanations that bring together evidence from different areas of neuroscience are based. As we explain, current criteria for evaluating such evidence to determine its admissibility in legal contexts are inadequate. Appealing to literature in the philosophy of scientific experimentation and theoretical work in the social, cognitive and behavioral sciences, we lay the groundwork for reforming these criteria and identify some of the implications of modifying them. (shrink)

While first-person methods are essential for a science of consciousness, it is controversial what form these methods should take and whether any such methods are reliable. I propose that first-person experiments are a reliable method for investigating conscious experience. I outline the history of these methods and describe their characteristics. In particular, a first-person experiment is an intervention on a subject's experience in which independent variables are manipulated, extraneous variables are held fixed, and in which the subject makes a phenomenal (...) judgement about the target experience of the investigation. I examine historical and contemporary examples of first-person experiments: Mariotte’s demonstration of the visual blind spot, Kanizsa’s subjective contours, the Tse Illusion, and investigations of the non-uniform resolution of the visual field. I discuss the role that phenomenal contrast plays in these methods, and how they overcome typical introspective errors. I argue that their intersubjective repeatability is an important factor in their scientific status, however, it is not the only factor. That they control for extraneous factors and confounds is another factor which sets them apart from pseudoscience (e.g., the perception of auras), and hence another reason for classifying them as genuine experiments. Furthermore, by systematically mapping out the structure of visual experience, these methods make scientific progress. Praises of such first-person experimental approaches may not always be sung by philosophers and psychologists, but they continue to flourish as respectable scientific methods nevertheless. (shrink)

Background We report a randomised controlled clinical trial of neurofeedback therapy intervention for ADHD/ADD in adults. We focus on internal mechanics of neurofeedback learning, to elucidate the primary role of cortical self-regulation in neurofeedback. We report initial results; more extensive analysis will follow. Methods Trial has two phases: intervention and follow-up. The intervention consisted of neurofeedback treatment, including intake and outtake measurements, using a waiting-list control group. Treatment involved $\sim$40 hour-long sessions 2-5 times per week. Training involved either theta/beta or (...) sensorimotor-rhythm regimes, adapted by adding a novel 'inverse-training' condition to promote self-regulation. Follow-up (ongoing) will consist of self-report and executive function tests. Setting Intake and outtake measurements were conducted at University of Helsinki. Treatment was administered at partner clinic Mental Capital Care, Helsinki. Randomisation We randomly allocated half the sample then adaptively allocated the remainder to minimise baseline differences in prognostic variables. Blinding Waiting-list control design meant trial was not blinded. Participants 54 adult Finnish participants (mean age 36 years; 29 females) were recruited after screening by psychiatric review. 44 had ADHD diagnoses, 10 had ADD. Measurements Symptoms were assessed by computerised attention test (T.O.V.A.) and self-report scales, at intake and outtake. Performance during neurofeedback trials was recorded. Results Participants were recruited and completed intake measurements during summer 2012, before assignment to treatment and control, September 2012. Outtake measurements ran April-August 2013. After dropouts, 23 treatment and 21 waiting-list participants remained for analysis. Initial analysis showed that, compared to waiting-list control, neurofeedback promoted improvement of self-reported ADHD symptoms, but did not show transfer of learning to T.O.V.A. Comprehensive analysis will be reported elsewhere. Trial Registration "Computer Enabled Neuroplasticity Treatment (CENT)", ISRCTN13915109. Partly funded by Finnish science agency TEKES, project #440078. (shrink)

This essay offers a comprehensive vision for a higher education program incorporating critical thinking across the curriculum at hypothetical Alpha College, employing a rigorous detailed conception of critical thinking called “The Alpha Conception of Critical Thinking”. The program starts with a 1-year, required, freshman course, two-thirds of which focuses on a set of general critical thinking dispositions and abilities. The final third uses subject-matter issues to reinforce general critical thinking dispositions and abilities, teach samples of subject matter, and introduce subject-specific (...) critical thinking. Subject-matter departmental and other units will make long-range plans for incorporating critical thinking in varying amounts in subject-matter courses, culminating in a written Senior Thesis/Project involving investigating, taking, and defending a position, which reinforce critical thinking abilities and dispositions and increase subject-matter knowledge. Teaching approaches used in the program are involving and based on the principle, “We learn what we use.” Both summative and formative assessment are employed as appropriate. Coordination and support are extensive. Objections and concerns are discussed, and alternatives, including possible transitions, are considered. An extended review of research supports moving toward CTAC. (shrink)

Can new technology enhance purpose-driven, democratic dialogue in groups, governments, and societies? Online Deliberation: Design, Research, and Practice is the first book that attempts to sample the full range of work on online deliberation, forging new connections between academic research, technology designers, and practitioners. Since some of the most exciting innovations have occurred outside of traditional institutions, and those involved have often worked in relative isolation from each other, work in this growing field has often failed to reflect the full (...) set of perspectives on online deliberation. This volume is aimed at those working at the crossroads of information/communication technology and social science, and documents early findings in, and perspectives on, this new field by many of its pioneers. -/- CONTENTS: -/- Introduction: The Blossoming Field of Online Deliberation (Todd Davies, pp. 1-19) -/- Part I - Prospects for Online Civic Engagement -/- Chapter 1: Virtual Public Consultation: Prospects for Internet Deliberative Democracy (James S. Fishkin, pp. 23-35) -/- Chapter 2: Citizens Deliberating Online: Theory and Some Evidence (Vincent Price, pp. 37-58) -/- Chapter 3: Can Online Deliberation Improve Politics? Scientific Foundations for Success (Arthur Lupia, pp. 59-69) -/- Chapter 4: Deliberative Democracy, Online Discussion, and Project PICOLA (Public Informed Citizen Online Assembly) (Robert Cavalier with Miso Kim and Zachary Sam Zaiss, pp. 71-79) -/- Part II - Online Dialogue in the Wild -/- Chapter 5: Friends, Foes, and Fringe: Norms and Structure in Political Discussion Networks (John Kelly, Danyel Fisher, and Marc Smith, pp. 83-93) -/- Chapter 6: Searching the Net for Differences of Opinion (Warren Sack, John Kelly, and Michael Dale, pp. 95-104) -/- Chapter 7: Happy Accidents: Deliberation and Online Exposure to Opposing Views (Azi Lev-On and Bernard Manin, pp. 105-122) -/- Chapter 8: Rethinking Local Conversations on the Web (Sameer Ahuja, Manuel Pérez-Quiñones, and Andrea Kavanaugh, pp. 123-129) -/- Part III - Online Public Consultation -/- Chapter 9: Deliberation in E-Rulemaking? The Problem of Mass Participation (David Schlosberg, Steve Zavestoski, and Stuart Shulman, pp. 133-148) -/- Chapter 10: Turning GOLD into EPG: Lessons from Low-Tech Democratic Experimentalism for Electronic Rulemaking and Other Ventures in Cyberdemocracy (Peter M. Shane, pp. 149-162) -/- Chapter 11: Baudrillard and the Virtual Cow: Simulation Games and Citizen Participation (Hélène Michel and Dominique Kreziak, pp. 163-166) -/- Chapter 12: Using Web-Based Group Support Systems to Enhance Procedural Fairness in Administrative Decision Making in South Africa (Hossana Twinomurinzi and Jackie Phahlamohlaka, pp. 167-169) -/- Chapter 13: Citizen Participation Is Critical: An Example from Sweden (Tomas Ohlin, pp. 171-173) -/- Part IV - Online Deliberation in Organizations -/- Chapter 14: Online Deliberation in the Government of Canada: Organizing the Back Office (Elisabeth Richard, pp. 177-191) -/- Chapter 15: Political Action and Organization Building: An Internet-Based Engagement Model (Mark Cooper, pp. 193-202) -/- Chapter 16: Wiki Collaboration Within Political Parties: Benefits and Challenges (Kate Raynes-Goldie and David Fono, pp. 203-205) -/- Chapter 17: Debian’s Democracy (Gunnar Ristroph, pp. 207-211) -/- Chapter 18: Software Support for Face-to-Face Parliamentary Procedure (Dana Dahlstrom and Bayle Shanks, pp. 213-220) -/- Part V - Online Facilitation -/- Chapter 19: Deliberation on the Net: Lessons from a Field Experiment (June Woong Rhee and Eun-mee Kim, pp. 223-232) -/- Chapter 20: The Role of the Moderator: Problems and Possibilities for Government-Run Online Discussion Forums (Scott Wright, pp. 233-242) -/- Chapter 21: Silencing the Clatter: Removing Anonymity from a Corporate Online Community (Gilly Leshed, pp. 243-251) -/- Chapter 22: Facilitation and Inclusive Deliberation (Matthias Trénel, pp. 253-257) -/- Chapter 23: Rethinking the ‘Informed’ Participant: Precautions and Recommendations for the Design of Online Deliberation (Kevin S. Ramsey and Matthew W. Wilson, pp. 259-267) -/- Chapter 24: PerlNomic: Rule Making and Enforcement in Digital Shared Spaces (Mark E. Phair and Adam Bliss, pp. 269-271) -/- Part VI - Design of Deliberation Tools -/- Chapter 25: An Online Environment for Democratic Deliberation: Motivations, Principles, and Design (Todd Davies, Brendan O’Connor, Alex Cochran, Jonathan J. Effrat, Andrew Parker, Benjamin Newman, and Aaron Tam, pp. 275-292) -/- Chapter 26: Online Civic Deliberation with E-Liberate (Douglas Schuler, pp. 293-302) -/- Chapter 27: Parliament: A Module for Parliamentary Procedure Software (Bayle Shanks and Dana Dahlstrom, pp. 303-307) -/- Chapter 28: Decision Structure: A New Approach to Three Problems in Deliberation (Raymond J. Pingree, pp. 309-316) -/- Chapter 29: Design Requirements of Argument Mapping Software for Teaching Deliberation (Matthew W. Easterday, Jordan S. Kanarek, and Maralee Harrell, pp. 317-323) -/- Chapter 30: Email-Embedded Voting with eVote/Clerk (Marilyn Davis, pp. 325-327) -/- Epilogue: Understanding Diversity in the Field of Online Deliberation (Seeta Peña Gangadharan, pp. 329-358). -/- For individual chapter downloads, go to odbook.stanford.edu. (shrink)

Recent advances in causal reasoning have given rise to a computational model that emulates the process by which humans generate, evaluate, and distinguish counterfactual sentences. Contrasted with the “possible worlds” account of counterfactuals, this “structural” model enjoys the advantages of representational economy, algorithmic simplicity, and conceptual clarity. This introduction traces the emergence of the structural model and gives a panoramic view of several applications where counterfactual reasoning has benefited problem areas in the empirical sciences.

Przedmowa Problematyka związana z zależnościami przyczynowymi, ich modelowaniem i odkrywa¬niem, po długiej nieobecności w filozofii i metodologii nauk, budzi współcześnie duże zainteresowanie. Wiąże się to przede wszystkim z dynamicznym rozwojem, zwłaszcza od lat 1990., technik obli¬czeniowych. Wypracowane w tym czasie sieci bayesowskie uznaje się za matematyczny język przyczynowości. Pozwalają one na daleko idącą auto¬matyzację wnioskowań, co jest także zachętą do podjęcia prób algorytmiza¬cji odkrywania przyczyn. Na potrzeby badań naukowych, które pozwalają na przeprowadzenie eksperymentu z randomizacją, standardowe metody ustalania zależności przyczynowych (...) opracowano na początku XX wieku. Zupełnie inaczej sprawa przedstawia się w przypadku badań nieeksperymentalnych, gdzie podobne rozwiązania pozostają kwestią przyszłości. Zadaniem tej książki jest podanie warunków, które powinny być spełnione przez te rozwiązania, oraz sformułowanie proceduralnego kryterium zależności przy¬czynowych jako szczegółowej realizacji tych warunków. Pociąga ono waż¬kie konsekwencje dla filozofii i metodologii nauk, które ujawnia – podany w Części II – zarys me-todolo¬gii proceduralnej. W literaturze przedmiotu brakuje w miarę wszechstronnego i systema¬tycznego omówie¬nia najnowszych filozoficznych i metodologicznych dys¬kusji na temat przy¬czynowości, co niech będzie wytłumaczeniem, dlaczego w niektórych punktach obecnej książki szczegółowo referuję trudno dos¬tępne teksty źró¬dłowe. Przymiotnik „proceduralny” używam tu w znaczeniu węższym niż Huw Price (w którego pracach właściwszy byłby termin „kryterialny”) dla podkre¬ślenia – zgodnie z łacińskim źródłosłowem procedo – że dla ustalenia przy¬czyny niezbędne jest podjęcie przez uczonych określonych interakcji z ba¬daną rzeczywistością. Zalążki zamysłu prezentowanego w tej książce przedstawiłem podczas warsztatów filozoficznych „Philosophy and Probability” w roku 2002, zor¬ganizowanych przez Instytut Filozofii Uniwersytetu w Konstancji. Wdzięczny jestem uczestnikom tych warsztatów za uwagi, a przede wszyst¬kim następującym osobom: Luc Bovens, Brandon Fitelson, Alan Hájek, Stephan Hartmann oraz Jon Williamson. Podczas międzynarodowej konferencji „Analytical Pragmatism”, zorgani¬zowanej w Lublinie w roku 2003 przez Wydział Filo¬zofii Katolickiego Uniwersytetu Lubelskiego, odniosłem swoją koncepcję do prac Nancy Cartwright. Szczególnie inspirujący okazał się komentarz Huw Price’a do mojego referatu i przeprowadzona z nim dysku¬sja. Ujęcie koncepcji metodologii proceduralnej na szerszym tle współ-czes¬nego nurtu empirystycznego w filozofii nauki przedstawiłem w roku 2004 podczas konferencji „5th Quadrennial Fellows Conference”, zorgani¬zowanej przez Instytut Filozofii Uniwersytetu Jagiellońskiego oraz Centrum Filozofii Nauki w Pittsburghu. Szczególnie pomocne w dalszych moich pra¬cach były uwagi Jamesa Bogena, Janet Kourany, Jamesa Lennoxa, Johna Nortona, Thomasa Bonka, Jana Woleńskiego i Johna Worralla, za które wyra¬żam swoją wdzięczność. Korpus książki powstał podczas mojego stażu w Centrum Filozofii Nauki w Pittsburghu, który odbyłem jako stypendysta Fundacji na Rzecz Nauki Polskiej w roku akademickim 2004-2005. Uczestniczyłem w tym czasie w życiu naukowym Centrum i w pracach badawczych zespołu z Instytutu Filozofii Uniwersytetu Carnegie-Mellon w Pittsburghu kierowanego przez Clarka Glymoura. Na jego ręce składam podziękowanie za wiele po¬mocnych uwag do moich wy¬stąpień oraz tekstów i za dyskusje przede wszystkim z nim samym i z jego najbliższymi współpracownikami: Peterem Spirtesem oraz Richardem Scheinesem, a także pozostałymi członkami tego zes¬połu, doktorantami i uczestnikami seminarium badawczego „Causality in the Social Sciences”. Za wieloletnie wsparcie, wielopłaszczyznowe inspiracje towarzyszące pi¬saniu tej książki, a także liczne pomocne uwagi do jej wcześniejszych wersji dziękuję przede wszystkim Księdzu Profesorowi Andrzejowi Bronkowi oraz Księdzu Profesorowi Józefowi Herbutowi, współprowadzącemu seminarium doktorskie w Katedrze Metodologii Nauk Katolickiego Uniwersytetu Lubel¬skiego im. Jana Pawła II, jak również pozostałym uczestnikom tego semina¬rium. Dziękuję mojej Żonie, dr Annie Kawalec za wiele wysiłku włożonego w ulepszenie redakcji – językowej i merytorycznej – obecnej książki. Książkę tę można czytać na kilka sposobów. Czytelnikom zainteresowa¬nym przede wszystkim prowadzeniem badań empirycznych polecałbym rozpoczęcie od Rozdziału 2. i kontynuację pozostałych rozdziałów Części I, a następnie Dodatków. Czytelnikom zainteresowanym problemami filozo¬fii i metodologii nauk polecałbym rozpoczęcie lektury książki od Części II i uzupełniającą lekturę Rozdziału 2., a następnie Wprowadzenia i Zakończenia. Czytelnikom mniej zainteresowanym zagadnieniami teoretycznymi pole¬całbym zapoznanie się z fascynującymi dziejami odkrycia przyczyn cholery przez Johna Snowa, które rekonstruuję w Rozdziale 1. W dalszej części nato¬miast polecałbym przejście do Wprowadzenia i Zakończenia, które w mniej specjalistyczny sposób przybliżają proponowane tu rozstrzygnięcia. Tekst książki nie był dotąd publikowany. Wyjątkiem są pewne fragmenty Rozdziału 8. oraz 9., które w zmienionej postaci ukazały się w Rocznikach Filozoficznych (Kawalec 2004). -/- Lublin, luty 2006 r. (shrink)

The need to align multiple experimental procedures and produce converging results so as to demonstrate that the phenomenon under investigation is real and not an artifact is a commonplace both in scientific practice and discussions of scientific methodology (Campbell and Stanley 1963; Wimsatt 1981). Although sometimes this is the purpose of aligning techniques, often there is a different purpose—multiple techniques are sought to supply different perspectives on the phenomena under investigation that need to be integrated to answer the questions scientists (...) are asking. After introducing this function, I will illustrate it by considering three of the major techniques in cognitive neuroscience for linking cognitive function with neural structure. (shrink)

The author asks whether there was a “scientific ‘68”, and focuses on aspects of two specific methodological proposals defined in the 1940s and 50s by the terms “action research” and “mixing methods”, applied particularly to social sciences. In the first, the climate surrounding the events of 1968 contributed to heightening the participative element to be found –by definition– in “action research”; that is: the importance of making the research subjects themselves participants in the design, execution and application of the study (...) of which they are the focus. This approach captured the democratic and anti-authoritarian spirit at the heart of the proposal, which was part of the prevailing climate in those days. The repercussions of 1968 on “mixing methods” focused on studying what had actually occurred, especially between the youth and workers, and therefore, particularly from the point of view of sociology and social psychology, using a “mixed methods” approach. The author explores the proposal of Norman Denzin; but traces the recent origins of both “mixing methods” and “action research” back to the proposals of mainly Kurt Lewin and the Chicago School. (shrink)

Hierarchies of evidence are an important and influential tool for appraising evidence in medicine. In recent years, hierarchies have been formally adopted by organizations including the Cochrane Collaboration [1], NICE [2,3], the WHO [4], the US Preventive Services Task Force [5], and the Australian NHMRC [6,7]. The development of such hierarchies has been regarded as a central part of Evidence-Based Medicine, a movement within healthcare which prioritises the use of epidemiological evidence such as that provided by Randomised Controlled Trials. Philosophical (...) work on the methodology of medicine has so far mostly focused on claims about the superiority of RCTs, and hence has largely neglected the questions of what hierarchies are, what assumptions they require, and how they affect clinical practice. This thesis shows that there is great variation in the hierarchies defended and in the interpretations they are, and can be, given. The interpretative assumptions made in using hierarchies are crucial to the content and defensibility of the underlying philosophical commitments concerning evidence and medical practice. Once this variation is been identified, it becomes clear that the little philosophical work that has been done so far affects only some hierarchies, under some interpretations. Modest interpretations offered by La Caze [11], conditional hierarchies like GRADE [12-14], and heuristic approaches such as that defended by Howick et al. [15,16] all survive previous philosophical criticism. This thesis extends previous criticisms by arguing that modest interpretations are so weak as to be unhelpful for clinical practice; that GRADE and similar conditional models omit clinically relevant information, such as information about variation in treatments’ effects and the causes of different responses to therapy; and that heuristic approaches lack the necessary empirical support. The conclusion is that hierarchies in general embed untenable philosophical assumptions: principally that information about average treatment effects backed by high-quality evidence can justify strong recommendations, and that the impact of evidence from individual studies can and should be appraised in isolation. Hierarchies are a poor basis for the application of evidence in clinical practice. The Evidence-Based Medicine movement should move beyond them and explore alternative tools for appraising the overall evidence for therapeutic claims. (shrink)

Investing in research is a long-term, risky proposition. In agriculture, it could take fifteen years or more for a research finding to show an improvement in a farmer’s field. Yet, research institutions, like other organizations it needs to be evaluated. For more than twenty years, independent panels of outside experts have evaluated each of the international research centers that the Consultative Group of International Agricultural Research (CGIAR) supports. This paper examines the evolution of this review system, outlines the key methodological (...) challenges faced, and draws lessons for others engaged in evaluating research institutions. It notes that the scope of the CGIAR reviews have been broadened over time in response to users’ concerns. Reviews now cover four dimensions of performance: research results, quality and relevance of research, vision and strategic directions, and management efficiency. The methodological challenges faced in measurement, valuation, and attribution are reviewed, along with practices found to be helpful in addressing these concerns. The paper concludes that the panel approach to institutional evaluation has served CGIAR’s needs well, and recommends it as an evaluation technique for integrating quantitative and qualitative dimensions of institutional performance. (shrink)

In this paper I look at causality in the context of intervention research, and discuss some problems faced in the evaluation of causal hypotheses via interventions. I draw attention to a simple problem for evaluations that employ randomized controlled trials. The common alternative to randomized trials, the observational study, is shown to face problems of a similar nature. I then argue that these problems become especially acute in cases where the intervention is complex (i.e. that involves intervening in a complex (...) system). Finally, I consider and reject a possible resolution of the problem involving the simulation of complex interventions. The conclusion I draw from this is that we need to radically reframe the way we think about causal inference in complex intervention research. (shrink)

A shared problem across the sciences is to make sense of correlational data coming from observations and/or from experiments. Arguably, this means establishing when correlations are causal and when they are not. This is an old problem in philosophy. This paper, narrowing down the scope to quantitative causal analysis in social science, reformulates the problem in terms of the validity of statistical models. Two strategies to make sense of correlational data are presented: first, a 'structural strategy', the goal of which (...) is to model and test causal structures that explain correlational data; second, a 'manipulationist or interventionist strategy', that hinges upon the notion of invariance under intervention. It is argued that while the former can offer a solution the latter cannot. (shrink)

External validity is typically regarded as the downside of case study research by methodologists and social scientists; case studies, however, are often aimed at drawing lessons that are generalizable to new contexts. The gap between the generalizability potential of case studies and the research goals demands closer scrutiny. I suggest that the conclusion that case study research is weak in external validity follows from a set of assumptions that I term the "traditional view," which are disputable at best. In this (...) view, external validity is treated as a matter of mere representativeness. I argue that it is best understood instead as a problem of inference and that the emphasis should be placed on the comparability of the study rather than on the typicality of the case. By making case studies highly comparable, their external validity can be reliably and efficiently assessed and, in this way, their generalizability potential enhanced. (shrink)

Computer simulations show that an unstructured neural-network model [Shultz, T. R., & Bale, A. C. (2001). Infancy, 2, 501–536] covers the essential features␣of infant learning of simple grammars in an artificial language [Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Science, 283, 77–80], and generalizes to examples both outside and inside of the range of training sentences. Knowledge-representation analyses confirm that these networks discover that duplicate words in the sentences are nearly identical and that they (...) use this near-identity relation to distinguish sentences that are consistent or inconsistent with a familiar grammar. Recent simulations that were claimed to show that this model did not really learn these grammars [Vilcu, M., & Hadley, R. F. (2005). Minds and Machines, 15, 359–382] confounded syntactic types with speech sounds and did not perform standard statistical tests of results. (shrink)

This article analyzes the impact of the rights-oriented business ethics course on student's ethical orientation. This approach, which is predominant in business schools, excludes the care-oriented approach used by a majority of women as well as some men and minorities. The results of this study showed that although students did not shift significantly in their ethical orientation, a majority of the men and an even greater majority of the women were care-oriented before and after a course in business ethics. If (...) business schools are to address society's increasing diversity then the perspective of women and others who are care-oriented must be assimilated into the curriculum. This can only be done by rethinking how the business ethics course (and the entire business curriculum) are taught to include a care-oriented approach. (shrink)

Researchers are concerned about whether manipulations have the intended effects. Many journals and reviewers view manipulation checks favorably, and they are widely reported in prestigious journals. However, the prototypical manipulation check is a verbal (rather than behavioral) measure that always appears at the same point in the procedure (rather than its order being varied to assess order effects). Embedding such manipulation checks within an experiment comes with problems. While we conceptualize manipulation checks as measures, they can also act as interventions (...) which initiate new processes that would otherwise not occur. The default assumption that manipulation checks do not affect experimental conclusions is unwarranted. They may amplify, undo, or interact with the effects of a manipulation. Further, the use of manipulation checks in mediational analyses does not rule out confounding variables, as any unmeasured variables that correlate with the manipulation check may still drive the relationship. Alternatives such as nonverbal and behavioral measures as manipulation checks and pilot testing are less problematic. Reviewers should view manipulation checks more critically, and authors should explore alternative methods to ensure the effectiveness of manipulations. (shrink)

How are we to appraise new technological developments that may bring revolutionary social changes? Currently this is often done by trying to predict or anticipate social consequences and to use these as a basis for moral and regulatory appraisal. Such an approach can, however, not deal with the uncertainties and unknowns that are inherent in social changes induced by technological development. An alternative approach is proposed that conceives of the introduction of new technologies into society as a social experiment. An (...) ethical framework for the acceptability of such experiments is developed based on the bioethical principles for experiments with human subjects: non-maleficence, beneficence, respect for autonomy, and justice. This provides a handle for the moral and regulatory assessment of new technologies and their impact on society. (shrink)

Reliability and external validity are two fundamental values that pose incompatible constraints on neurobiological experiments. The more reliability an experimental result achieves, the less external validity it earns, and vice versa. In this article, I propose an externalist interpretation of external validity: the external validity of an experimental result depends not only on how much complexity is built into an experimental design, but also on the relationship between the experimental result and other related experiments. This externalist interpretation, which explains how (...) a neurobiological experiment can be reliable and externally valid simultaneously, suggests a new way to understand the epistemology of neurobiological experimentation. (shrink)

The linguistic and cognitive sciences have severely underestimated the degree of linguistic diversity in the world. Part of the reason for this is that we have projected assumptions based on English and familiar languages onto the rest. We focus on some distortions this has introduced, especially in the study of semantics.

Despite an extensive amount of research studying the influence of significant others on an individual's ethical behavior, researchers have not examined this variable in the context of organizational group boundaries. This study tests actual and perceptual sharing and variation in ethical reasoning and moral intent within and across functional groups in an organization. Integrating theory on ethical behavior, group dynamics, and culture, it is proposed that organizational structure affects cognitive structure. Departmental boundaries create stronger social ties within the group as (...) well as intergroup biases between the groups. Thus individuals will be more likely to share in ethical reasoning and moral intent with members of their own functional group (in-group) than with members of other functional groups (out-group). Additionally, they will perceive that they are more likely to share in ethical reasoning and moral intent with in-group members than with out-group members. Responding to two versions of two ethical scenarios, respondents contrasted their own ethical behavior to their expected ethical behavior of in-group and out-group members. Empirical results confirmed the hypotheses. Organizational group boundaries create actual as well as perceptual sharing and variation in ethical reasoning and moral intent. Furthermore, when comparing perceptual sharing to actual sharing, results show that individuals understate their sharing of ethical reasoning and moral intent with out-group members and overstate their sharing with in-group members. As organizational boundaries can create actual and perceived differences between groups that could lead to inter-group conflict, suggestions for management focus on removing or blurring inter-group boundaries. (shrink)

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way (...) in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed. (shrink)

Recent interest in the external validity of prediction models has produced many methods for finding predictive distributions that are invariant to dataset shifts and can be used for prediction in new, unseen environments. However, these methods consider different types of shifts and have been developed under disparate frameworks, making it difficult to theoretically analyze how solutions differ with respect to stability and accuracy. Taking a causal graphical view, we use a flexible graphical representation to express various types of dataset shifts. (...) Given a known graph of the data generating process, we show that all invariant distributions correspond to a causal hierarchy of graphical operators, which disable the edges in the graph that are responsible for the shifts. The hierarchy provides a common theoretical underpinning for understanding when and how stability to shifts can be achieved, and in what ways stable distributions can differ. We use it to establish conditions for minimax optimal performance across environments, and derive new algorithms that find optimal stable distributions. By using this new perspective, we empirically demonstrate that that there is a tradeoff between minimax and average performance. (shrink)

The purpose of this research report is twofold: First, we analyze a complex of attitudes about rape myths, adversarial sexual beliefs, and gender-role conservatism; and second, we evaluate the impact of rape-education intervention strategies on American college students' attitudes. Using the Solomon four-group design, we randomly assigned 14 classes of Sociology 101 students to three different treatment conditions: a live rape-education workshop, a video of the workshop, and a control group. We found significant gender differences in students' attitudes on all (...) the scales, with women being more knowledgeable about rape, less likely to blame the victim, and less accepting of adversarial sexual beliefs and gender-role conservatism. Most important, we found that within the limits of the study, rape-education intervention works in changing some attitudes about rape for both men and women students. We examine the impact of the different educational strategies and explore curricular implications, including the need to teach about rape within a feminist context. (shrink)

Solidarity corresponds to virtuous social behavior, including personal freedom and responsibility, civic friendship, benevolence, reciprocity, and cooperation. These attributes are fundamentally good for individual persons and communities of work. Solidarity is therefore vitally important to the practice of humanistic management. This paper aims to provide management insights into the cultivation of solidarity. The paper begins by developing a theoretical framework to understand solidarity in business context, with attention to philosophical and theological connotations. An empirical research model is presented in the (...) form of a survey instrument to test for indications of solidarity in the workplace, and this measure is used to test several hypotheses regarding the positive associations of solidarity with validated measures of workplace engagement. Research results show that it is possible to identify and analyze workplace behaviors associated with solidarity. Data analysis confirms the validity of the model and demonstrates the positive associations of the hypotheses, based on empirical study of 40 workplaces and 399 employees. Moreover, the condition of whether the workplace was a public or private organization was found to affect the main relationship between solidarity and workplace engagement. The paper concludes with suggestions for practicable, tangible workplace behaviors based on the model, offering guidance in the pursuit of humanistic management. (shrink)

The idea that deep brain stimulation (DBS) induces changes to personality, identity, agency, authenticity, autonomy and self (PIAAAS) is so deeply entrenched within neuroethics discourses that it has become an unchallenged narrative. In this article, we critically assess evidence about putative effects of DBS on PIAAAS. We conducted a literature review of more than 1535 articles to investigate the prevalence of scientific evidence regarding these potential DBS-induced changes. While we observed an increase in the number of publications in theoretical neuroethics (...) that mention putative DBS-induced changes to patients’ postoperative PIAAAS, we found a critical lack of primary empirical studies corroborating these claims. Our findings strongly suggest that the theoretical neuroethics debate on putative effects of DBS relies on very limited empirical evidence and is, instead, reliant on unsubstantiated speculative assumptions probablyin lieuof robust evidence. As such, this may reflect the likelihood of a speculative neuroethics bubble that may need to be deflated. Nevertheless, despite the low number of first-hand primary studies and large number of marginal and single case reports, potential postoperative DBS changes experienced by patients remain a critical ethical concern. We recommend further empirical research in order to enhance theoretical neuroethics work in the area. In particular, we call for the development of better instruments capable of capturing potential postoperative variations of PIAAAS. (shrink)

How has sociology framed places of knowledge production and what is the specific power of the laboratory for this history? This article looks in three steps at how sociology and Science and Technology Studies (STS) have historically framed the world as laboratory. First, in early sociology, the laboratory was an important metaphor to conceive of sociology as a scientific enterprise. In the 1950s, the trend reversed and with the emergence of a ‘qualitative sociology’, sociology was seen in opposition to laboratory (...) work. With the ascent of laboratory studies, the laboratory perspective was again applied to many fields, including sociology itself. Based on a definition of a laboratory as aiming at placeless knowledge and being inconsequential this article argues that the two waves of laboratorization were metaphorical and did not really turn the world into a laboratory. Instead, two alternative concepts, those of the unilatory and the locatory, are proposed to gain a more precise understanding of some of these metaphorical uses of the term ‘laboratory’. (shrink)

Experiments are generally thought of as actions or operations undertaken to test a scientific hypothesis in settings detached from the rest of society. In this paper a different notion of experiment will be discussed. It is an understanding that has been developed in the classical tradition of the Chicago School of Sociology since the 1890s, but has so far remained unexplored. This sociological understanding of experiment does not model itself strictly on the natural sciences. Rather, it implies a process of (...) societal self-experimentation without a fixed setting of a sociological experimenter. The paper discusses this notion of experiment in relation to the recursive dependency of the application and the production of sociological knowledge. It is contended that this concept of a self-experimental society offers theoretical insights that could well prove fruitful for a sociological concept of experiment beyond the realm of the laboratory. (shrink)

Allan Franklin has identified a number of strategies that scientists use to build confidence in experimental results. This paper shows that Franklin's strategies have direct analogues in the context of computer simulation and then suggests that one of his strategies—the so-called 'Sherlock Holmes' strategy—deserves a privileged place within the epistemologies of experiment and simulation. In particular, it is argued that while the successful application of even several of Franklin's other strategies (or their analogues in simulation) may not be sufficient for (...) justified belief in results, the successful application of a slightly elaborated version of the Sherlock Holmes strategy is sufficient. (shrink)

In this paper I ask how educational researchers can believe the subjective perceptions of qualitative participant-observers given the concern for objectivity and generalisability of experimental research in the behavioural and social sciences. I critique the most common answer to this question within the educational research community, which posits the existence of two (or more) equally legitimate epistemological paradigms—positivism and constructivism—and offer an alternative that places a priority in educational research on understanding the purposes and meanings humans attribute to educational practices. (...) Only within the context of what I call a transcendent view from somewhere—higher ideals that govern human activities—can we make sense of quantitative as well as qualitative research findings. (shrink)

These preprints were automatically compiled into a PDF from the collection of papers deposited in PhilSci-Archive in conjunction with the PSA 2018.

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

Studies of knowledge utilization in public policy-making have important practical and theoretical implications. Accordingly, a voluminous work has been done on understanding and explaining the process of knowledge utilization (see Rich and Oh, 1993). However, we can easily find that there is the conspicuous absence of a greatly expanded understanding of the use of knowledge from those studies (Mandell and Sauter, 1984). Taken as a whole, empirical studies in the area of knowledge utilization have suffered from several critical problems (see (...) Rich, 1991; Oh, 1996a). More importantly, there appears to be serious conceptual and methodological gaps which need to be filled. This article focuses on one of the most significant methodological problems: measuring “knowledge utilization” from an evaluation perspective and offers some suggestions for future studies. (shrink)

The experimental revolution in the social sciences is one of the most significant methodological shifts undergone by the field since the ‘quantitative revolution’ in the nineteenth century. One of the often valued features of social science experimentation is precisely the fact that there are clear methodological rules regarding hypothesis testing that come from the methods of the natural sciences and from the methodology of RCTs in the biomedical sciences, and that allow for the adjudication among contentious causal claims. We examine (...) critically this claim and argue that some current understandings of the practices that surround social science experimentation overestimate the degree to which experiments can actually fulfil this role as “objective” adjudicators, by neglecting the importance of shared background knowledge or assumptions and of consensus regarding the validity of the constructs involved in an experiment. We take issue with the way the distinction between internal and external validity is often used to comment on the inferential import of experiments, used both among practitioners and among philosophers of science. We describe the ways in which the more common use of the internal/external distinction differs from Cook and Campbell’s original methodological project, in which construct validity and the four-fold validity typology were all important in assessing the inferential import of experiments. We argue that the current uses of the labels internal and external, as applied to experimental validity, help to encroach a simplistic view on the inferential import of experiments that, in turn, misrepresents their capacity to provide objective knowledge about the causal relations between variables. (shrink)

The aim of this study was to find out whether the achievement of high school social studies students differs significantly according to whether they are taught by the reflective teaching method or the lecture approach method. A stratified random sample of 215 year three social studies students was randomly assigned to two groups: 107 students into the reflective approach group and 108 into the lecture approach group. An achievement test was used as the pre‐ and the post‐test. The means of (...) the aggregate mean performance scores were compared by the use of Student's t‐test. There was no significant difference between the two groups on the pre‐test scores but there was a significant difference between the post‐test scores, in favour of the group taught by the reflective approach method. (shrink)