Back to Case Studies
Back to Case Studies
Location
Mountain View, CA
AI transparency is an emerging field in machine learning that aims to open up the black box nature of AI systems. This area inspects and communicates the steps and models involved in making algorithmic decisions. 1 For ethical, legal and business reasons, stakeholders–including data scientists, engineers, and business users–need to understand what their AI system is deciding, the machine learning (ML) algorithms behind it, what assumptions it relies on, and potential implications of each assumption. While the field is nascent, Google has undertaken the Model Cards initiative to address and improve industry-wide AI transparency
Many mature industries have developed standardized methods of benchmarking various systems under different conditions…By contrast, despite the broad reach and impact of machine learning models, there are no standardized formats to report the results of [ML] tests.2
Trained machine learning models are increasingly used to perform high impact tasks such as determining crime recidivism rates and predicting health risks. Researchers have discovered systematic biases in commercial machine learning models used for face detection and tracking, attribute detection, criminal justice, and other applications. However, these systematic errors were only exposed after models were put into use, and negatively affected users reported their experiences.2
Subsequently, a 2019 paper by Google researchers recommended an intervention that would clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well-suited, before the ML model is deployed. Specifically, the team proposed a framework for ‘model cards’ to encourage data scientists and other audiences/consumers to proactively read and understand possible outcomes and implications of using specific models. Model cards are short documents accompanying trained machine learning models that provide details and results related to a variety of conditional tests. These tests may relate to cultural, demographic, or phenotypic subgroups (e.g., race, geographic location, sex, Fitzpatrick skin tone) and intersectional subgroups (e.g., age and race, or sex and Fitzpatrick skin tone) that are relevant to the intended application domains.
“Model cards” use a standardized format for describing the essential facts about a machine learning model including its capabilities, limitations, performance metrics, and even ethical considerations. One of a model card’s most important functions is drawing boundaries around a model’s capabilities and limitations. How can model developers find these boundaries in a reliable, systematic way? A model’s performance can be measured in countless ways, but which are most relevant and useful? Likewise, which of a model’s potential limitations are most useful to quantify?
The answers to these questions—envisioned by model cards—have the potential to help investigate issues like unfair bias. For instance, does a model perform consistently across a diverse range of people, or does it vary in unintended ways as characteristics like skin color or region change? Model cards can bring clarity to important disparities, encouraging developers to consider their impact on a diverse range of people from the start of their model’s development process and keep these impacts in mind throughout.
Transparency in AI should be a common good. That’s why model cards aren’t intended to be a Google product, but a shared, evolving framework shaped by a range of voices. This includes users, developers, civil society stakeholders, and companies across the industry, as well as collaborations with organizations like Partnership on AI and their ABOUT ML Project.
Model cards have received significant attention from researchers, policymakers, and the media alike. These institutions include OpenAI, Salesforce, Intel, HSBC and government agencies including the US Department of Defense’s Joint Artificial Intelligence Center and the National Institute of Health. Most recently, Partnership on AI and OpenAI decided to apply Model Cards approach to their frameworks and systems, respectively. 3
As the framework is in early stages of development and adoption, future plans include: refining and scaling the model cards frameworks, easing the creation flow for Model Cards, and partnering with organizational leadership within Google and externally to drive broad-scale creation and adoption of Model Cards.
¹ Ron Schmelzer, Understanding Explainable AI (Forbes, July 2019) ↩
² Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019 (FAT* ‘19). ↩ ↩2
³ Reena Jana, From Boogle to Google - Meg Mitchell’s Mission to make AI for Everyone (Google Blog, January 2020) ↩
