Abstract

In 2012, a major traffic safety organization tasked the MIT AgeLab with developing a data-driven system for rating the effectiveness of new technologies intended to improve safety. Such a system was envisioned as having the potential to educate and guide consumers towards more confident and strategic purchasing decisions, ideally encouraging adoption of technologies with demonstrated safety benefit. In addition, an evaluation of the status and extent of existing data was seen as a way of identifying research gaps in the state of knowledge about safety systems. The focus was on technologies as a class, not on a rating review of individual vehicle model implementations. As conceptualized, the system aimed to complement traditional NCAP style ratings as well as to provide consumers with transparent information on early stage and often improving safety technologies. Development of the rating system and identification of data was undertaken in consultation with a range of academic, industrial, consumer, NGO, and governmental experts as well as with representatives of many of the major automotive manufacturers and suppliers. A key observation that emerged was that data on objectively demonstrable real-world benefits were generally sparse and often lower than expectations based on theoretical considerations, simulation studies, or pre-production evaluations. A number of experts and industry representatives expressed some surprise at both the divergence between theoretical and observed benefits and the relative scarcity of data upon which to make objective assessments, while others were quite aware of these issues and the need for the development of objective data under real-world operating conditions. A number of factors that might be relevant to understanding why such differences between expected and observed benefits exist were identified. One outcome of this effort was the founding of the Advanced Vehicle Technology (AVT) consortium to collect and examine objective data under naturalistic driving conditions of how drivers interact with, engage or don’t engage, various production safety and driver assistance systems. This ongoing effort is contributing to insights concerning actual benefits and reasons for benefit gaps. Drawing from our initial work, as well as newer sources of data, we argue that the evaluation and rating of safety and driver assistance technologies for informing the consumer and the public at large should consider both theoretical potential and existing demonstrated benefit of specific technologies. This position is increasingly relevant as the effectiveness of many newer technologies have the potential to actually improve over the lifecycle of a vehicle through software updates. The emphasis on ratings based on observed benefit for actual drivers under real-world conditions is proposed to be complementary, rather than competing with, ratings focused largely on controlled test track evaluations of engineered capability. In addition, a case is made for providing ratings that assesses benefit relative to overall crash, injury, and fatality rates – and in relation to the specific scenario / crash event type that a given technology is intended to address. This approach should aid consumers in considering the extent to which a specific technology is or is not relevant to their particular driving needs.