Back to Case Studies
Back to Case Studies
Location
San Francisco, CA
Designed with leading civil rights organizations like Color Of Change, Upturn, AAJC, NAACP, and more, Project Lighthouse is an anonymized research approach to help determine when and where racial discrimination happens on the Airbnb platform and the effectiveness of policies that fight it. Project Lighthouse maintains the privacy of sensitive demographic data to prevent its exploitation while providing Airbnb users with agency over whether they participate in the program. Through its novel privacy and inclusion-centric approach, Project Lighthouse aspires to elevate building for equity as a standard for responsible innovation in the minds of technologists.
Project Lighthouse is a groundbreaking initiative launched in the United States to uncover, measure, and help overcome discrimination when booking or hosting on Airbnb.
Following a civil rights audit in 2016, Airbnb took steps to reduce discrimination on their platform, including the creation and enforcement of a strict Non-Discrimination Policy, removal of guest profile photos from the booking process, and the increase of listings available for instant booking, among other changes. Since 2016, over 1.4 million people have declined to agree to Airbnb’s mandatory Community Commitment and Non-Discrimination Policy, and have been removed from the platform. However, because Airbnb does not ask its users for racial demographic data, the company struggles to measure the effectiveness of these policies or how else people may be discriminated against on their platform.
Airbnb, like online platforms and marketplace rental services in general, needs data and tools to combat discrimination effectively. But studying demographic data opens a risk of exploitation by external actors with nefarious intentions or internal actors inadvertently causing harm.
Consequently, the main challenge in developing a system to study discrimination on the platform is twofold: (1) designing a privacy-centric process that utilizes anonymized perceived race data and (2) trying to ensure that the resultant data can be used to accurately measure potential experience gaps in the product.
Project Lighthouse was designed with input from leading civil rights and privacy organizations to thoughtfully measure discrimination while respecting individual privacy.
Project Lighthouse was developed with input from leading civil rights groups and privacy organizations, including Color Of Change, AAJC, Center for Democracy & Technology, The Leadership Conference on Civil & Human Rights, LULAC, the NAACP, National Action Network, and Upturn. In designing a privacy-centric process that utilizes anonymized perceived race data, Project Lighthouse employs p-sensitive k-anonymity to measure experience gaps by perceived race.
K-anonymity means that there are at least k instances of each unique set of values (e.g. for columns number_of_accepts, number_of_rejects used in the technical paper for discussion) in Airbnb’s dataset, which can be achieved in a number of ways, such as by averaging comparable number_of_accepts. For instance, number_of_accepts values of 2, 3, and 5 could all be represented as 3.33, thereby removing a unique identifier that could be used to infer the perceived race of a user. K refers to the minimum number of users represented by each unique set of values in the anonymized dataset, or the degree of anonymization. In the above example, k=3 as the 3 individuals’ data can no longer be distinguished after the K-anonymization process. In complement to k-anonymity, p-sensitive k-anonymity means that each unique set of values in Airbnb’s dataset has at least p distinct perceived race values among them. P-sensitive k-anonymity further protects against the risk of uniquely identifying data in the case of all k users having the same perceived race.
This privacy-centric approach works in concert with A/B testing and other research to support the understanding and mitigation of discrimination that may occur on Airbnb’s platform in a way that protects user-perceived demographic data. Additionally, Project Lighthouse grants users agency over their privacy as a user can opt-out at any time
Project Lighthouse was in development for two years before being publicly launched in the United States in June 2020. Since September 2020, the Project Lighthouse team has been analyzing collected data to inform future design iterations of platform facets.
The ultimate aspiration of Project Lighthouse is to create a genuinely equitable experience for all users on the Airbnb platform and to elevate the concept of building for equity across all tech companies as a core component of responsible innovation. As a first step, Airbnb has shared its methodology behind Lighthouse in a publicly available technical paper in the hopes that the paper can be a starting point for further innovations that rigorously measure discrimination while upholding user privacy. However, Airbnb acknowledges that Lighthouse is tailored to its platform and not agnostic to the size and structure of other technology companies. Therefore, Airbnb is exploring inventing additional methodologies to provide directional information for technology companies of various sizes to combat discrimination.
