Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models

Proceedings of the 12th Conference on Language Resources and Evaluation (2020)
Download Edit this record How to cite View on PhilPapers
Abstract
As the demand for explainable deep learning grows in the evaluation of language technologies, the value of a principled grounding for those explanations grows as well. Here we study the state-of-the-art in explanation for neural models for natural-language processing (NLP) tasks from the viewpoint of philosophy of science. We focus on recent evaluation work that finds brittleness in explanations obtained through attention mechanisms.We harness philosophical accounts of explanation to suggest broader conclusions from these studies. From this analysis, we assert the impossibility of causal explanations from attention layers over text data. We then introduce NLP researchers to contemporary philosophy of science theories that allow robust yet non-causal reasoning in explanation, giving computer scientists a vocabulary for future research
Categories
PhilPapers/Archive ID
GRIWAI-2
Upload history
Archival date: 2020-06-23
View other versions
Added to PP index
2020-06-23

Total views
105 ( #39,351 of 58,375 )

Recent downloads (6 months)
72 ( #9,612 of 58,375 )

How can I increase my downloads?

Downloads since first upload
This graph includes both downloads from PhilArchive and clicks on external links on PhilPapers.