Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models

Proceedings of the 12th Conference on Language Resources and Evaluation (2020)
Download Edit this record How to cite View on PhilPapers
Abstract
As the demand for explainable deep learning grows in the evaluation of language technologies, the value of a principled grounding for those explanations grows as well. Here we study the state-of-the-art in explanation for neural models for natural-language processing (NLP) tasks from the viewpoint of philosophy of science. We focus on recent evaluation work that finds brittleness in explanations obtained through attention mechanisms.We harness philosophical accounts of explanation to suggest broader conclusions from these studies. From this analysis, we assert the impossibility of causal explanations from attention layers over text data. We then introduce NLP researchers to contemporary philosophy of science theories that allow robust yet non-causal reasoning in explanation, giving computer scientists a vocabulary for future research
Categories
PhilPapers/Archive ID
GRIWAI-2
Revision history
Archival date: 2020-06-23
View upload history
References found in this work BETA
Causation, Prediction, and Search.Spirtes, Peter; Glymour, Clark & Scheines, Richard

View all 34 references / Add more references

Citations of this work BETA

No citations found.

Add more citations

Added to PP index
2020-06-23

Total views
12 ( #50,415 of 50,067 )

Recent downloads (6 months)
12 ( #38,358 of 50,067 )

How can I increase my downloads?

Downloads since first upload
This graph includes both downloads from PhilArchive and clicks to external links.