United States Government Accountability Office Report to Congressional Addressees September 2021 TECHNOLOGY ASSESSMENT Exposure Notification Benefits and Challenges of Smartphone Applications to Augment Contact Tracing GAO-21-104622 The cover image displays a stylized depiction of the use of an exposure notification app by various individuals and an example of an exposure notification message. Cover source: GAO. | GAO-21-104622 TECHNOLOGY ASSESSMENT Highlights of GAO-21-104622, a report to congressional addressees Exposure Notification September 2021 Benefits and Challenges of Smartphone Applications to Augment Contact Tracing Why GAO did this study What GAO found With the emergence and rapid global Exposure notification applications (apps)—which determine the proximity of users spread of COVID-19, smartphone and notify people who have been in close contact with another user who was likely apps have been developed to supplement manual contact tracing, infectious—are expected to enhance the speed and reach of contact tracing and which is a public health measure used help slow the spread of infectious diseases such as COVID-19. As of June 2021, to slow the spread of infectious almost half (26/56) of U.S. states, territories, and the District of Columbia had disease. deployed an app for COVID-19, all using a system developed jointly by Google and GAO was asked to conduct a Apple (see figure). In the absence of a national app, states independently launched technology assessment of exposure apps, resulting in a staggered rollout over 10 months beginning in August 2020. notification apps. This report Map of deployment of exposure notification apps by U.S. states and territories, as of discusses (1) the benefits of exposure notification apps; (2) the current level June 2021 of deployment in the U.S.; (3) challenges affecting their use; and (4) policy options that may help address these challenges for future use. To address these objectives, GAO reviewed agency documentation, met with officials from several federal agencies, and conducted a review of technical and policy literature. GAO also interviewed representatives from companies involved in the development of exposure notification apps, public health organizations, federally funded research and development centers, and academic Reported app development costs for selected states varied, ranging from no cost researchers. In addition, GAO (provided by a nonprofit organization) to $700,000. Marketing costs for selected analyzed information from a selection states ranged from $380,000 to $3.2 million. Reported app download levels in the of states. GAO is identifying policy selected states ranged from 200,000 to more than 2 million, as of June 2021. options in this report. GAO identified several challenges limiting app use and the ability of states and GAO received technical comments on others to determine whether the apps were effective: a draft of this report from five federal agencies and five organizations Accuracy of Technical limitations to measuring distance and exposure can result in included in the review, which it measurements inaccurate exposure notifications. incorporated as appropriate. Privacy and The public may lack confidence that its privacy is being protected, in security concerns part, due to a lack of independent privacy and security assessments and a lack of federal legal protections. Adoption States have faced challenges attracting public interest in downloading and using an exposure notification app. Verification code States faced challenges in promptly providing people who tested delays positive for COVID-19 with a verification code necessary to notify other close contacts of potential exposure using the app. Evidence of Limited data are available to evaluate the effectiveness of the apps. effectiveness View GAO-21-104622. For more information, Source: GAO. | GAO-21-104622 contact Karen L. Howard at (202) 512-6888 or, howardk@gao.gov or Vijay A. D’Souza, at (202) 512-6240, dsouzav@gao.gov. United States Government Accountability Office GAO developed the following four policy options that could help address challenges related to exposure notification apps. The policy options identify possible actions by policymakers, which may include Congress, other elected officials, federal agencies, state and local governments, and industry. See below for details of the policy options and relevant opportunities and considerations. Policy Options to Help Address Challenges of Exposure Notification Apps for Future Use Opportunities Considerations Research and Development • Research on technological limitations could help • The research needed may be costly. (report page 41) increase accuracy, encouraging users to download • Improvements may not be cost-effective, Policymakers could promote and use the apps. since existing apps may already be research and development to • Research on technologies and architectures other sufficiently accurate. address technological than those used by U.S. states could lead to • Research may result in apps that are not limitations. improvements. functional for the next pandemic, since the • Partnerships with technology companies could spur current apps were developed for COVID-19. innovation and help with integrating improvements. Privacy and Security • Uniform standards and best practices could help • Policymakers would need to balance the Standards and Practices address real and perceived risks to the public’s data, need for privacy and security with the costs (report page 42) potentially increasing adoption. of implementing standards and practices. Policymakers could promote • Standards developed by a broad coalition of • Implementation of privacy requirements uniform privacy and security stakeholders could increase the likelihood of may need to be flexible, since jurisdictions standards and practices for stakeholder agreement and buy-in. could use different approaches. exposure notification apps. • Standards and practices could be challenging to oversee and enforce. Best Practices (report page • Best practices could help authorities better promote • Best practices could require consensus from 43) app adoption. many public- and private-sector Policymakers could promote • Best practices could help state public health stakeholders, which can be time- and best practices for authorities by providing information on procedures resource-intensive. approaches to increasing and potential approaches for distributing • Current best practices may have limited adoption and to measure verification codes in a timely manner. relevance to a future pandemic. the effectiveness of • Best practices could help public health authorities • In some cases, stakeholders may lack exposure notification apps. establish a more rigorous way to measure the sufficient information or the experience to extent of app use and any resulting improvements develop best practices. in notifying exposed people. National Strategy (report • Enhanced national coordination that builds on • A coordinated national approach would page 44) the underlying infrastructure and lessons likely have associated costs and require Policymakers could learned from COVID-19 could prompt faster sustained funding during the pandemic. collaborate to enhance deployment of apps in the future. • Coordination of groups with divergent the pandemic national • A future national marketing campaign with perspectives and interests may pose strategy and promote a cohesive and coherent messaging could result challenges to defining outcomes, measuring coordinated approach in wider adoption. performance, and establishing a leadership to the development • Policymakers could recommend a national app that approach. and deployment of public health authorities could decide to use based • It is unclear whether potential users exposure notification on their individual needs. A national app could add would be more or less likely to trust a apps. more functions by integrating exposure notification national exposure notification app than capabilities with test scheduling and vaccine one developed by a state government. delivery coordination. Source: GAO. | GAO-21-104622 This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Table of Contents Introduction ........................................................................................................................ 1 1 Background ...................................................................................................................... 4 1.1 COVID-19 ........................................................................................................................... 4 1.2 Manual contact tracing for COVID-19 ............................................................................... 4 1.3 Roles of states, federal agencies, and other stakeholders ................................................ 6 2 Benefits and Design of Exposure Notification Apps ........................................................ 9 2.1 Exposure notification apps are expected to provide enhanced speed and reach ............ 9 2.2 How exposure notification apps work ............................................................................. 10 2.3 Apps can use either a centralized, decentralized, or hybrid system to manage data.....12 2.4 States are widely using the Google and Apple Exposure Notifications system ..............14 3 Deployment and Adoption of Exposure Notification Apps ........................................... 18 3.1 About half of the states have deployed an exposure notification app ........................... 18 3.2 About half of the states use a customized app ............................................................... 21 3.3 States reported that app development time and cost varied based on several factors .............................................................................................................. 25 3.4 Officials reported varying download levels and use........................................................ 26 4 Challenges Associated with Exposure Notification Apps ............................................... 27 4.1 Accuracy of measurements ............................................................................................. 27 4.2 Privacy and security concerns ......................................................................................... 28 4.3 Adoption and use of apps ................................................................................................ 31 4.4 Verification code delays .................................................................................................. 32 4.5 Limited evidence of effectiveness ................................................................................... 33 5 Policy Options That Could Help Address Challenges for Future Use ............................. 41 5.1 Policy option: Research and development ...................................................................... 41 5.2 Policy option: Privacy and security standards and best practices ................................... 42 5.3 Policy option: Best practices to measure effectiveness .................................................. 43 5.4 Policy option: Enhance the national strategy .................................................................. 44 Exposure Notification GAO-21-104622 i 6 Agency and Expert Comments ....................................................................................... 46 Appendix I: Objectives, Scope, and Methodology ............................................................ 47 Appendix II: Exposure Notification App Adoption Rates for Selected U.S. States ........... 51 Appendix III: GAO Contacts and Staff Acknowledgments ................................................ 54 Exposure Notification GAO-21-104622 ii Abbreviations APHL Association of Public Health Laboratories BLE Bluetooth Low Energy CDC Centers for Disease Control and Prevention COVID-19 Coronavirus Disease 2019 DHS Department of Homeland Security FCC Federal Communications Commission FTC Federal Trade Commission GPS Global Positioning System HHS Health and Human Services NIH National Institutes of Health NIST National Institute of Standards and Technology QR quick response WHO World Health Organization Exposure Notification GAO-21-104622 iii 441 G St. N.W. Washington, DC 20548 Introduction September 9, 2021 Congressional Addressees For more than a century, public health authorities have used contact tracing to track and limit the spread of infectious diseases. Manual contact tracing involves interviewing infected people to identify others they have been in contact with, notifying those contacts that may have been exposed, and advising the infected individual and contacts to take appropriate measures. Manual contact tracing can be effective, but it has limitations. Specifically, it is a resource- intensive process, and it is most effective during the early stage of an outbreak, when case numbers tend to be lower, or during phases with fewer cases. 1 In addition, its effectiveness relies, in part, on prompt and complete identification of individuals and notification of contacts, which could be difficult with a rapidly spreading disease. Other limitations of manual contact tracing include the reliance on human recall of contacts and movements (which can be prone to error), and the inability to identify strangers. Further, people may not be forthcoming in sharing information about their contacts. With the emergence and rapid spread of the highly infectious Coronavirus Disease 2019 (COVID- 19), digital contact tracing technologies have been developed to supplement manual contact tracing and help address its limitations. One such technology is a type of application (app) developed for use on a smartphone. 2 Referred to as an exposure notification app, 3 it is intended to be used to notify a smartphone user who has been in close contact with another user who later tested positive for COVID-19. 4 This type of app is intended to reduce transmission by notifying potentially exposed people faster than manual contact tracing, including contacts the infected person may not have known. 1 Center for Infectious Disease Research and Policy, COVID-19: the CIDRAP Viewpoint, (Minneapolis, Minn.: University of Minnesota, June 2, 2020). 2 An estimated 85 percent of adults in the U.S. own a smartphone, according to a survey conducted by the Pew Research Center. However, this decreases to an estimated 61 percent for people 65 and older. In addition, the rate varies based on other factors, including income level and whether a person lives in an urban or rural area. This estimate was based on a survey of U.S. adults, conducted between Jan. 25 and Feb. 8, 2021. See Pew Research Center, Mobile Fact Sheet, (Washington, D.C.: Apr. 7, 2021), accessed July 1, 2021, https://www.pewresearch.org/internet/fact-sheet/mobile. 3 An exposure notification app can include both software that a user downloads to use on a smartphone, or a function built into the phone’s operating system that can be activated by users. 4 While contact tracers have previously used smartphone apps, including for data entry, we are unaware of any exposure notification apps in public use prior to the COVID-19 pandemic. Exposure Notification GAO-21-104622 1 In July 2020, we issued a Science & Tech Spotlight overview of exposure notification app technology. 5 Since then, there has been an increase in the development and use of these apps. You asked us to conduct an assessment of these technologies. This report discusses (1) the benefits and design of exposure notification apps, (2) the current level of deployment in the U.S., (3) challenges affecting their use, and (4) policy options that could help address key challenges for future use. To address all of these objectives, we reviewed documentation and met with officials from selected federal agencies involved in providing guidance, funding research, and directing other efforts related to exposure notification apps. In addition, we interviewed representatives from entities involved in the development of exposure notification apps; public health organizations; federally funded research and development centers; academic researchers; and nongovernmental organizations. We also conducted a review of literature discussing exposure notification apps, including their benefits, design, and challenges, as well as relevant policy options. To identify the current level of deployment, we developed an inventory of exposure notification apps that had been deployed by U.S. states, territories, and the District of Columbia (hereafter referred to as states) as of June 2021. States that had an app in a pilot phase at the time of our review were included the category of “states that had not deployed an app as of June 2021.” To obtain additional information associated with the development and use of these apps, we interviewed state public health officials from a non-generalizable sample of nine states that had deployed an exposure notification app as of January 1, 2021: Alabama, Colorado, Connecticut, Minnesota, Nevada, North Carolina, Pennsylvania, Virginia, and Washington. We also reached out to two other states (Louisiana and Utah) that deployed apps in the later stages of our evidence collection and received written feedback to structured questions about the status of their efforts to deploy an app, their rates of adoption, and other topics. We selected the sample of nine states to obtain a range of views, based on factors such as deployment date, geographical distribution, number of COVID-19 cases, and app developer. Because the selection was based on a non-generalizable sample, the results were not used to make inferences about all states that had deployed an app. In addition to our interviews with officials from the selected states, we conducted a review of each of the selected states apps, including the key functions, features, and privacy use policies for those apps. To obtain perspectives from states that had not deployed an app, we collected information from a non-generalizable selection of seven states that had not deployed an app at the time of our review (Montana, Nebraska, Oregon, Rhode Island, South Carolina, Texas, and West Virginia). 5 GAO, Science & Tech Spotlight: Contact Tracing Apps, GAO-20-666SP (Washington, D.C.: July 28, 2020). While the terms exposure notification apps and contact tracing apps have been used to describe these technologies, we will use the term exposure notification apps in this review. Exposure Notification GAO-21-104622 2 For these states, we conducted an interview with officials from one state and obtained written responses to a semi-structured set of questions for the other six. We identified policy options that may address the identified challenges based on our literature review and interviews. We assessed each policy option by identifying potential benefits and considerations of implementing them, as identified over the course of our review. See appendix I for a detailed description of our objectives, scope, and methodology. We conducted our work from November 2020 to September 2021 in accordance with all sections of GAO’s Quality Assurance Framework that are relevant to technology assessments. The framework requires that we plan and perform the engagement to obtain sufficient and appropriate evidence to meet our stated objectives and to discuss any limitations to our work. We believe that the information and data obtained, and the analysis conducted, provide a reasonable basis for any findings and conclusions in this product. Exposure Notification GAO-21-104622 3 1 Background 1.1 COVID-19 nation. 7 In the U.S., there have been more than 596,000 reported deaths 8 and 32 The outbreak of COVID-19 was first million reported confirmed and probable reported on December 31, 2019, in Wuhan, cases as of July 2021. 9 In addition, despite China. 6 In the weeks that followed, the virus strides made in getting people vaccinated, quickly spread around the globe. On the threat of variants is growing, including January 31, 2020, the Secretary of Health evidence of increased transmissibility. As a and Human Services declared a public result, uncertainty about the future of the health emergency for the U.S., retroactive COVID-19 pandemic remains. to January 27, which followed a World Health Organization (WHO) declaration on 1.2 Manual contact tracing for January 30 that the outbreak constituted a public health emergency of international COVID-19 concern. On March 11, 2020, WHO characterized the COVID-19 outbreak as a Contact tracing is a key component in global pandemic due to its levels of spread controlling the transmission and spread of and its severity. COVID-19 is highly infectious diseases, according to the contagious and may be spread by people Centers for Disease Control and Prevention who are not showing symptoms (i.e., (CDC). 10 Contact tracing is intended to “asymptomatic”) or before symptoms separate the people who have (or may appear (“pre-symptomatic”). have) an infectious disease from those who do not and provide information on other More than a year later, as we have measures the potentially exposed contacts should take, such as being tested for the previously reported, the pandemic has disease or self-isolating. Together, the test, resulted in catastrophic loss of life and trace, and isolate strategy is part of the substantial damage to the global economy, and to the stability and security of our 6 9 This disease is caused by SARS-CoV-2 (Severe Acute Data on COVID-19 cases in the U.S. are based on aggregate Respiratory Syndrome, Coronavirus 2). case reporting to the Centers for Disease Control and 7 Prevention, COVID Data Tracker, (Atlanta, Ga.), accessed July GAO, COVID-19: Key Insights from the GAO’s Oversight of 7, 2021, the Federal Public Health Response, GAO-21-396T https://covid.cdc.gov/covid-data-tracker/#datatracker-home (Washington, D.C.: Feb. 24, 2021). , and include probable and confirmed cases as reported by 8 states and jurisdictions. Centers for Disease Control and CDC’s National Center for Health Statistics COVID-19 death counts in the U.S. are based on provisional counts from Prevention (CDC) COVID-19 counts are subject to change death certificate data, which do not distinguish between due to delays or updates in reported data from states and laboratory-confirmed and probable COVID-19 deaths. territories. According to CDC, the actual number of COVID- Provisional death counts are incomplete due to an average 19 cases is unknown for a variety of reasons, including that delay of 2 weeks (a range of 1–8 weeks or longer) for death people who have been infected may not have been tested or certificate processing. Data include deaths occurring from may not have sought medical care. January 2020 through the week ending on July 3, 2021. 10 Centers for Disease Control and Prevention, Operational Centers for Disease Control and Prevention, National Center Consideration for Adapting a Contact Tracing Program to for Health Statistics, (Atlanta, Ga.), accessed July 7, 2021, Respond to the COVID-19 Pandemic in non-US Settings, https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm. (Atlanta, Ga.: June 23, 2021). Exposure Notification GAO-21-104622 4 broader effort to limit the transmission of agencies typically maintain an existing infectious diseases such as COVID-19. capacity to conduct contact tracing for infectious diseases, this capacity is generally Contact tracers are the people who sufficient only to respond to relatively small manually trace the contacts of each person or isolated outbreaks. who has tested positive for COVID-19. Contact tracers begin the process by Contact tracing is resource intensive, interviewing the person with the positive because, as cases increase, the contact test result in order to identify others whom tracer will need more time to contact all that person might have contacted. Next, potentially exposed persons. Hence, more the tracer advises the person and the and more tracers will be needed to ensure contacts to take containment measures comprehensive contact tracing of all (e.g., a 14-day quarantine for COVID-19), diagnosed cases and potentially exposed and coordinates or provides information on persons. The particular features of the any needed care, testing recommendations, COVID-19 pandemic—asymptomatic and resources. 11 persons and the ability to spread rapidly— require a significantly large workforce of For COVID-19, CDC defines a close contact contact tracers. According to the National as anyone who has been within 6 feet of an Association of County and City Health infected person for a total of 15 minutes or Officials, the benchmark rate is 30 contact more over a 24-hour period (for example, tracers per 100,000 people. This equates to three individual 5-minute exposures for a about 98,460 contact tracers needed to total of 15 minutes). 12 According to CDC, cover the entire U.S. population. 13 infected persons can spread COVID-19 According to CDC, state health departments starting from 48 hours (or 2 days) before reported a total of 51,855 employed they have symptoms or test positive for contact tracers for the month of December COVID-19. 2020, which was about one month before the peak of reported new cases in the U.S. 14 In a public health emergency such as the COVID-19 pandemic, it is critical that each To supplement the capabilities of manual state has a sufficient workforce of contact contact tracing, several states have used tracers in order to contain the disease. smartphone apps. These apps include those Although state and local public health that help people monitor their COVID-19 11 13 Consistent with CDC guidance, except in certain National Association of County and City Health Officials, circumstances, people who have been in close contact with Position Statement: Building COVID-19 Contact Tracing someone who has COVID-19 should quarantine. However, Capacity in Health Departments to Support Reopening people who have been fully vaccinated and people who American Society Safely (Washington, D.C.: Apr. 16, 2020). were previously diagnosed with COVID-19 within the last 3 14 months may not need to quarantine. This number represents a best estimate for a 1-month snapshot and may not include contact tracers employed at 12 See Centers for Disease Control and Prevention, the local health department or community level, according “Appendices,” COVID-19, (Atlanta, Ga.: Updated July 2, to CDC documentation. Data are reported monthly, and 2021), accessed July 2, 2021, estimates will continue, and be updated regularly, according https://www.cdc.gov/coronavirus/2019-ncov/php/contact-t to CDC. We used the reported estimates from December racing/contact-tracing-plan/appendix.html#contact. 2020 to illustrate capacity just before the peak cases observed in January 2021. Exposure Notification GAO-21-104622 5 symptoms, assist people in recalling the Federal agencies places they had visited when providing that information to a contact tracer, and Federal agencies—including CDC, the exposure notification apps. 15 Department of Homeland Security (DHS), the National Institutes of Health (NIH), and 1.3 Roles of states, federal agencies, the National Institute of Standards and and other stakeholders Technology (NIST)—have taken various steps to assist states in the development Various entities have a role in the and use of exposure notification apps, deployment and use of exposure including issuing guidance, distributing notification apps within the U.S. These funds to states, and funding research. entities include states, federal agencies, and other stakeholders, such as national public Specifically, in May 2020 and December health organizations and organizations 2020, CDC issued two guidance documents involved in research and development of on digital contact tracing tools, which the apps. included discussion of exposure notification apps. 16 The guidance is intended to provide States health departments with minimum and preferred characteristics of the apps, including those for contact notification and In the U.S., public health authorities at the data security. 17 In addition, CDC distributes state, territorial, and local levels plan and funds to states—through established coordinate pandemic response actions mechanisms such as its Epidemiology and within their jurisdictions. In addition, these Laboratory Capacity for Prevention and authorities generally lead contact tracing Control of Emerging Infectious Diseases efforts, including the implementation of cooperative agreement, which currently related technologies, such as exposure provides funds to 64 jurisdictions to detect, notification apps. prevent, and respond to the growing threats posed by infectious diseases, including for the development and use of exposure notification apps, according to 15 16 For example, in April 2020, Utah deployed an app which Centers for Disease Control and Prevention, Preliminary allowed residents to check their symptoms, and privately Criteria for the Evaluation of Digital Contact Tracing Tools share a subset of their location information with public for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and health officials to aid in the contact tracing process. In Guidelines for the Implementation and Use of Digital Tools to summer 2020, Utah disabled the location–based services in Augment Traditional Contact Tracing, version 1.0 (Atlanta, this app. Ga: Dec. 15, 2020). 17 These characteristics included that the apps should enable health departments to define different exposure risk levels used to identify contacts based on how close and how long their exposure was and to require user consent before their data are shared with a health department. Exposure Notification GAO-21-104622 6 CDC documentation. 18 (See ch. 3 for years. NIH has also funded various projects additional information on uses of this related to contact tracing tools. 20 funding.) In early 2020, NIST began work on a project, CDC has also funded research on exposure which is currently ongoing, to study and notification apps, including research develop exposure notification systems with performed by the Massachusetts Institute strong privacy and cybersecurity of Technology’s Lincoln Laboratory, to protocols. 21 As a part of this project, NIST examine barriers to adoption and the held an event in June 2020 to help facilitate efficacy of the underlying technologies used research aimed at improving the by various apps. Further, CDC officials performance of these kinds of apps. stated that the agency has provided Further, in January 2021, NIST held a ongoing support and consultation to states workshop on challenges associated with interested in implementing exposure exposure notification apps. notification apps. For example, since early August 2020, CDC has coordinated with Other stakeholders Lincoln Laboratory to host meetings with state public health authorities where they National public health organizations have can discuss issues related to app issued guidance and provided other support development and deployment. to state public health authorities to assist in the development and deployment of In addition, DHS’s Science and Technology exposure notification apps. These Directorate provided funding to two organizations include the Association of projects through its Silicon Valley Public Health Laboratories (APHL), Innovation Program. 19 These projects are Association of State and Territorial Health intended to develop criteria the apps can be Officials, Council of State and Territorial tested against and to enable the capability Epidemiologists, National Association of to test the apps using the criteria. According County and City Health Officials, Linux to DHS officials, they expect that these Foundation Public Health, the Public Health projects will be completed in the next 2 Informatics Institute, and others. Other key stakeholders include entities involved in the research and development of exposure 18 19 The 64 jurisdictions receiving awards under the DHS intends for this program to find new technologies Epidemiology and Laboratory Capacity for Prevention and that strengthen national security. Control of Emerging Infectious Diseases cooperative 20 agreements include all 50 states, several large metro areas, These projects included tools to identify businesses and and U.S. territories and affiliates. A full list of recipients is hot spots visited by people with COVID-19 and development provided on the Centers for Disease Control and of a digital health pass to enable businesses to verify health Prevention’s website. See Centers for Disease Control and credentials. Prevention, National Center for Emerging and Zoonotic 21 A system (or protocol) provides a framework that Infectious Diseases, Division of Preparedness and Emerging determines the function of a particular software application, Infections, Recipients, Project Officers, and Jurisdictional like an app on a smartphone. Assignment Listing, (Atlanta, Ga.: last reviewed June 16, 2021), accessed June 30, 2021, https://www.cdc.gov/ncezid/dpei/elc/advisor-list.html. Exposure Notification GAO-21-104622 7 notification apps or analysis of their they developed this system to help performance. Specifically, researchers, governments and the global community organizations, and technology companies slow the spread of the COVID-19 pandemic. have played a key role in the design of the In addition, Google and Apple collaborated systems used by exposure notification with Microsoft and APHL to establish and apps. 22 host servers to facilitate the system. Further, Google and Apple collaborated In May 2020, Google and Apple—the two with the MITRE Corporation to deploy the primary developers of operating systems Exposure Notification Private Analytics for smartphones—collaborated on the portal, which provides public health development of an exposure notification authorities with data on the performance of system used by the states discussed later in the states’ apps. This effort involves several the report. According to Google and Apple, other partners, including the Internet Security Research Group and NIH. 22 These include, for example, the TCN Coalition and Massachusetts Institute of Technology Pact developed systems—referred to as the Temporary Contact Numbers Protocol, or TCN Protocol; and the Private Automated Contact Tracing (PACT) protocol, respectively. Exposure Notification GAO-21-104622 8 2 Benefits and Design of Exposure Notification Apps Exposure notification apps are intended to documentation, and publications that we automate and augment the manual contact reviewed. tracing process, with enhanced speed and reach being among the expected benefits, Speed. Apps are expected to allow for according to scientific literature, state faster identification and notification of officials, federal agency documents, and contacts. After a positive test result is representatives of stakeholder received, apps automate the process of organizations we interviewed. They work by identifying and notifying contacts. This using proximity detection to determine automation can lead to faster notification, when two smartphone users are in close which in turn can lead to faster changes in contact, then notifying all contacts of a user individual behavior aimed at helping slow who later reports a positive test result for disease transmission, namely testing and quarantine, according to selected studies. 23 COVID-19. The apps can use a centralized, decentralized, or hybrid system to collect, store, and manage data. Many states within Reach. Apps are also expected to provide the U.S. are using apps based on a more complete and faster identification of decentralized system that was developed contacts because, unlike manual contact jointly by Google and Apple. tracing, they do not rely on a person’s memory to identify the people they came into contact with, according to CDC 2.1 Exposure notification apps are documents. In addition, according to a Pew expected to provide enhanced Research Center report, 41 percent of speed and reach Americans asked about their views on speaking with a public health official Exposure notification apps are expected to reported that they are unlikely to talk with provide two key benefits—speed and reach. contact tracers. The report also noted that Specifically, they are expected to allow for younger adults, those with lower incomes, more timely identification and notification and those with less formal education are of contacts and greater (more complete) especially unlikely to engage with manual coverage of contacts, according to the contact tracers. 24 Apps may provide a way majority of the selected states, CDC to increase coverage of these populations. In addition, apps can reach people even 23 Institute of Technology, Oct. 29, 2020), and J.A. Moreno John Hopkins University and Association of State and Territorial Health Officials, A National Plan to Enable Lopez et al., “Anatomy of Digital Contact Tracing: Role of Comprehensive COVID-19 Case Finding and Contact Tracing Age, Transmission, Setting, Adoption, And Case Detection,” in the US (Baltimore, Md.: Johns Hopkins University, Apr. 10, Science Advances, vol. 7, no. 15 (2020): eabd8750. 2020), Massachusetts Institute of Technology Lincoln 24 Pew Research Center, The Challenges of Contact Tracing Laboratory, Realizing the Promise of Automated Contact as U.S. Battles COVID-19, (last updated Oct. 30, 2020), Tracing Technology to Control the Spread of COVID-19: accessed July 2, 2021, Recommendations for Smartphone App Deployment, Use, https://www.pewresearch.org/fact-tank/2020/10/30/key-fi and Iterative Assessment (Cambridge, Mass.: Massachusetts ndings-about-americans-views-on-covid-19-contact-tracing/. Exposure Notification GAO-21-104622 9 when manual contact tracing resources are signal can receive and store these limited or overwhelmed. encounter messages. The distance between two phones can be estimated 2.2 How exposure notification apps by comparing the strength of the BLE signal when it was sent with its strength work when it is received. 27 • If one or more of the messages later Exposure notification apps use proximity turns out to have been from a contact detection to determine whether two app who tested positive for COVID-19, a users are in close contact. The app then central server or a user’s smartphone notifies a person if they had been in close analyzes the encounter message to contact with another user who was likely determine whether the user’s risk of infectious at the time, and who voluntarily exposure exceeds a predetermined confirmed their diagnosis in the app. threshold. The risk analysis includes Proximity detection involves a series of factors such as the time spent at automated actions that determine the various distances and when the contact proximity of two persons, notify them of occurred in relation to when the potential exposure, and provide guidance in contact was most infectious. 28 the case of exposure. The formula used to calculate the level of The proximity detection steps are described risk, including the specific risk factors, can more fully here. be set by the public health agency. The assessment generally involves determining • An exposure notification app whether the encounter meets the CDC’s periodically broadcasts messages definition of a close contact (i.e., at least 15 (referred to as encounter messages) minutes within 6 feet within 24 hours). The using a wireless radio transmission apps do not consider other factors that technology—Bluetooth Low Energy affect the risk of infection. For example, (BLE) 25—that contain, among other they do not consider whether the users things, a random identifier and the were wearing masks, or whether the strength of the signal sent (i.e., transmitted power). 26 Any other phone encounter occurred in a well ventilated location (e.g., indoors or outdoors). that has the same or similar app installed and is in range of the user’s 25 27 BLE is a wireless radio transmission technology with a The BLE signal will weaken as the distance between two range of around 30 feet. BLE started to be included in smartphones increases. The strength of the signal when it is smartphones in 2011, and is now included on most received is referred to as a received signal strength smartphones to enable communication between devices, indication measurement. such as smart watches and wireless headphones. 28 This can be determined based on when the person first 26 To help preserve user privacy and to limit the ability to had symptoms or was tested. track the movements of other users, the random identifiers are changed on a periodic basis (e.g., every 10 minutes). Further, the identifiers do not reveal any personal information about other users. Exposure Notification GAO-21-104622 10 If a user’s risk of exposure exceeds the risk of measurements and other potential threshold, the user receives an exposure technologies. notification from the app. 29 The exposure notification can also include other Location data can also be used instead of, information, such as when the exposure or in addition to, the data gathered using occurred and the next steps the person the BLE messages. Location data are not should take, such as getting tested, currently used by U.S. states. However, monitoring symptoms, and self- other nations (e.g., Israel) have apps that quarantining. See figure 1 for an example of use Global Positioning System (GPS) data to an exposure notification message. track and record a person’s location, including the date and time. 30 Further, apps The distance measured between two can track user locations by having the user phones using BLE is only an estimate, and scan a quick response (QR) code at a its accuracy can be affected by various specific location (e.g., venue, restaurant). 31 factors. See section 4.1 for additional The app then records the location, date, discussion of factors affecting the accuracy 29 31 In certain systems, a public health provider could provide A QR code is a barcode with the ability to encode different the notification in lieu of an app notification. types of information. Each location needs to have a unique 30 QR code and it must be accessible (e.g., posted at the Other methods to determine a smartphone’s location entrance to a building). include assisted-GPS, the triangulation of cell towers, and Wi-Fi access point identification. Exposure Notification GAO-21-104622 11 and time. 32 See figure 2 for an example of a 2.3 Apps can use either a QR code. centralized, decentralized, or hybrid system to manage data Exposure notification apps can use a centralized, decentralized, or hybrid architecture for collecting, storing, and analyzing data. 35 The main difference between these types of architecture is the extent to which the information used to determine exposure is stored and analyzed on a central server or on an individual smartphone. These differences affect the privacy protections built into the system. A decentralized architecture may help preserve a user’s privacy more than a centralized architecture. These types of architecture are described more fully here. • In a centralized architecture, most of A user’s recent locations can be compared the data are stored on a central server, with a list of locations of people who have which also analyzes the data to tested positive for COVID-19 to determine determine which users may have been the risk of exposure. 33 However, GPS exposed. For example, the central location estimates are only accurate within server collects personal information as about a 16-foot radius outdoors. In a part of users’ registration and addition, the accuracy decreases near generates the random identifiers used buildings, bridges, and trees, and indoors or for the encounter messages. A public underground. 34 Thus, the location estimates health authority can access this may not always be reliable in determining information (including information on whether two people were in close contact. which users were in contact) and aggregate it to perform further analyses of the data to identify additional potential exposures and to identify potential surges in cases to help inform 32 35 The app uses the phone’s camera to scan the QR code. Data architecture is a framework that comprises of 33 models, policies, rules, and standards that govern the The list of locations from infected persons can also include collection, storage, arrangement, integration, and use of locations obtained through manual contact tracing. data in organizations. 34 National Coordination Office for Space-Based Positioning, Navigation, and Timing, GPS Accuracy, (Washington, D.C.: last update Apr. 22, 2020), accessed May 17, 2020, https://www.gps.gov/systems/gps/performance/accuracy. Exposure Notification GAO-21-104622 12 broader mitigation and response contact tracing efforts, and identifying efforts. The central server can also where infections may be occurring. For incorporate data from other sources, example, this architecture does not such as manual contact tracing (e.g., allow authorities to know who received locations an infected person visited). an exposure notification. However, storing data on a centralized • A hybrid architecture incorporates server can also reveal potentially aspects of both architectures. sensitive information to governmental Specifically, the random identifier organizations, or others who gain generation for encounter messages access to the server. An example of a remains decentralized (i.e., handled by centralized app is one used by the user smartphones) to help preserve nation of Singapore, which it deployed privacy, while the risk analysis and in March 2020. notifications are handled by the central • In a decentralized architecture, most of server. Hybrid systems have been the data are located on users’ developed by researchers, but we are smartphones, with only limited data on not aware of their use at a national or a central server. Each user’s device state level. analyzes the data to determine whether an exposure has occurred. This Table 1 provides a comparison of the approach may help preserve personal different architectures, including how data privacy; however, it also limits the data are managed and the key advantages and that are available to public health disadvantages to each approach. authorities for determining the effectiveness of the apps, informing Exposure Notification GAO-21-104622 13 Table 1: Advantages and disadvantages of centralized, decentralized, and hybrid data architectures used in exposure notification apps Centralized Decentralized Hybrid Where most information is central server smartphone device smartphone device stored Where random identifiers central server smartphone device smartphone device are generated Where exposure data are central server smartphone device central server analyzed Level of data access by higher lower moderate public health authorities Key advantages Public health authorities Seeks to preserve Seeks to preserve can access data to individual privacy by individual privacy and perform analysis, limiting data accessible to provides health identify additional entities (e.g., public health authorities with useful exposure and potential authorities). data. surges, and inform response efforts.a Key disadvantages Data could reveal Limits the data that are Data could reveal potentially sensitive available to public health potentially sensitive information to public authorities for determining information to public health authorities or how well the app works health authorities or other entities that gain and to inform response other entities that gain access to the server. efforts. access to the server. Source: Based on GAO review of technical and industry documentation. I GAO-21-104622 aThe central server can also incorporate data from other sources, such as manual contact tracing (which can provide the location of an infected person among other things). 2.4 States are widely using the Google customizing their own exposure notification and Apple Exposure Notifications apps. system In September 2020, Google and Apple provided public health authorities with an U.S. states with apps are using the Google and additional option (referred to as the Express Apple Exposure Notifications system. 36 (See option), which was intended to make it easier ch. 3 for additional information on for authorities to use the Google Apple deployment of apps by state public health system by removing the need for the authorities.) The Google Apple system was authorities to build their own custom apps. In released in May 2020 as an application- this option, Google developed an app for programming interface to be used by public Android-based phones, and Apple deployed health authorities in developing and app-less functionality, such that a person can 36 For the purposes of this report we refer to the Google and Apple Exposure Notifications system as the Google Apple system. Exposure Notification GAO-21-104622 14 enable the system for their area (if available) encounter messages. If there is a match, the through the settings on an iPhone. 37 app analyzes the risk of exposure based on the method and parameters established by The Google Apple system uses BLE and a the public health authority. If the risk exceeds decentralized architecture. In this system, a predetermined threshold, the app displays each user’s app creates a temporary key an exposure notification to the app user. The (changes every 24 hours) that the app uses to notification can include guidance and generate random identifiers, and to encrypt instructions. Figure 3 provides an overview of information provided in the encounter this process. messages. The app then exchanges a random identifier with other users’ apps, and In August 2020, Microsoft partnered with maintains a list of the encounter messages APHL to establish a key server that could be that the user has received. To help preserve used by all U.S. states—the National Key the privacy of the users, the encounter Server. With its launch, apps from different messages do not include personal information U.S. states could be interoperable, so that app or location data. users can find out if they have been exposed without needing to download and use apps If the user tests positive for COVID-19, a from multiple states. 38 This feature is public health authority uses a verification particularly important in regions where server to generate a verification code, and commuters regularly cross jurisdictional then sends the user the code to verify the boundaries (e.g., in neighboring areas of positive test result. A user can then Washington D.C., Maryland, and Virginia). voluntarily input this code in the app to Further, according to APHL, the server also submit their recent temporary keys (e.g., reduces the burden of each state’s public prior 14 days) to a key server. If a user health agency needing to build and host its chooses not to input the code in the app, the own key server. user will not enable the notification of other recent close contacts who were also using the In addition to the National Key Server, APHL app of potential exposure. manages a central verification server, referred to as the Multi-tenant Verification Each exposure notification app periodically Server, which was launched in September downloads the temporary keys from people 2020. APHL made the verification server who had recently tested positive from the key available to reduce the effort needed by server and then compares it with its list of public health agencies to bring an exposure 37 38 The Express option works differently on iPhones and Android App interoperability means that a person using an app from phones. For iPhones, the Express option is built into the one state could receive an exposure notification based on an operating system and can be activated by users in the iPhone encounter with any other person who had an app that also settings. To receive exposure notifications, no app is required used this server, such as a person from another state. and therefore it is said to have “app-less” functionality. For Android smartphones, states develop the configuration settings (e.g., risk parameters), and Google then develops the app, which states can then use. For Apple devices, the Google Apple system works on iPhones running at least iOS 12.5. For Android, the system works for any smartphone capable of running Android version 6. Exposure Notification GAO-21-104622 15 notification app to their jurisdiction. As a part of the Google Apple system, APHL noted that a verification server is necessary to ensure a user has received a positive test result before uploading their temporary keys to the National Key Server. APHL also noted that rather than each public health agency standing up its own verification server and deciding on a verification approach, providing one verification server reduces the time and cost to deploy the Google Apple system. For the states and territories with apps, nearly all were using the National Key Server, while over two-thirds were using the Multi-tenant Verification Server as of August 2021, according to APHL. 39 39 Although a verification server is necessary, it does not have to be the Multi-tenant Verification Server, so some states elected to use their own verification servers. Exposure Notification GAO-21-104622 16 Exposure Notification GAO-21-104622 17 3 Deployment and Adoption of Exposure Notification Apps Almost half of U.S. states have deployed an 3.1 About half of the states have exposure notification app. In the absence of a deployed an exposure notification national exposure notification app, states app have independently launched their own apps at different times, resulting in a staggered As of June 2021, 26 of 56 U.S. states rollout. States have developed and deployed (including territories and the District of apps using the Google Apple system and Columbia) have deployed an app (see fig. 4). about half have customized their apps, which Unlike other countries, the U.S. does not have provides the apps with more flexibility and a national exposure notification app; instead, functionality. According to officials from states have independently deployed selected states, development time, costs, and individual apps. levels of adoption have varied. Exposure Notification GAO-21-104622 18 The patchwork of app deployment shown in The 26 states deployed apps over a span of 10 figure 4 arises from the fact that public health months, in a staggered rollout beginning in authorities at the state and territorial levels August 2020. Figure 5 provides a timeline of decide whether and when to deploy an app. app deployment and related events. Virginia Furthermore, there was no existing option for was the first state to deploy an exposure a national app that states could use, notification app, in August 2020, and according to CDC documentation. As of June Massachusetts was the most recent, in June 2021, 26 out of 56 states had deployed apps. 2021. Seventeen of the states deployed an Officials from seven selected states that had app between October 2020 and June 2021, not deployed an app cited several reasons for which was after the Express option was made that decision, including limited cell phone available (see fig. 5). coverage in rural areas or other challenges related to the deployment and use of an app (see ch. 4 for additional detail). They also cited competing priorities, such as natural disaster response or vaccine distribution efforts, and were concerned that exposure notification app development would divert limited resources away from other priorities. Exposure Notification GAO-21-104622 19 Note: The timeline indicates the Google Apple exposure notification option that was initially deployed by the state (i.e., custom or Express). Seven states deployed the Express option after initially deploying a custom app (Minnesota, Nevada, Hawaii, New York, Virginia, Louisiana, and New Jersey) between January and April 2021. In addition to exposure notification apps based on the Google Apple system, a few states developed smartphone apps to help people monitor their COVID-19 symptoms or assist in recalling the places they had visited when providing that information to a contact tracer. For example, in April 2020, Utah deployed an app which allowed residents to check their symptoms, and privately share a subset of their location information with public health officials to aid in the contact tracing process. In summer 2020, Utah disabled the location-based services in this app. Exposure Notification GAO-21-104622 20 3.2 About half of the states use a customized app Of the 26 U.S. states that had deployed apps as of June 2021, all are using a version of the Google Apple system. 40 With this system, public health authorities can choose to develop customized apps, use the Express option, or use both in tandem (see table 2). States could use, for example, a customized app for Android and the Express option for iOS smartphones. 41 40 41 In addition to the U.S., most countries with an app use BLE One of the selected states uses a customized app for both (primarily using the Google Apple system); approximately Android and iOS and also enabled the Express option for iOS. one-third use GPS. Some countries use both BLE and QR codes; for example, the United Kingdom’s National Health Service’s app uses both the Google Apple system and QR codes to check-in to locations. Exposure Notification GAO-21-104622 21 Table 2: U.S. states deployment of apps using the Google Apple Exposure Notifications system States Customized app Express option Alabama ● ○ Arizona ● ○ California ○ ● Colorado ○ ● Connecticut ○ ● Delaware ● ○ District of Columbia ○ ● Guam ● ○ Hawaii ● ● Louisiana ● ● Maryland ○ ● Massachusetts ○ ● Michigan ● ○ Minnesota ● ● Nevada ● ● New Jersey ● ● New Mexico ○ ● New York ● ● North Carolina ● ○ North Dakota ● ○ Pennsylvania ● ○ Utah ○ ● Virginia ● ● Washington ○ ● Wisconsin ○ ● Wyoming ● ○ Total 16 17 Legend: ● = State or territory that is using a version of the Google and Apple Exposure Notifications system; ○ = State or territory that is not using the identified version of the Google Apple system. Source: GAO compilation of data from selected states, related documents, interviews, and other sources. I GAO-21-104622 Note: State public health authorities can deploy customized exposure notification apps, which may offer unique functions and features. State public health authorities can also elect to use the Express option of the Google Apple system. The Express option provides convenience and efficiency but potentially less flexibility to tailor exposure notification functionality. The total in the table (33) does not equal the number of states with apps (26) because some states use both a customized app and the Express option. To build a customized exposure notification university partners). For the Express option, app, public health authorities can seek public health authorities can provide Google external technical support (e.g., third-party and Apple an electronic configuration file that developers, nonprofit organizations, or includes instructions and content, including Exposure Notification GAO-21-104622 22 the risk parameters for enabling an exposure For example, a customized app may help notification and messaging for app users. users identify testing facilities and access While states can use their own internal state-level statistics about COVID-19 technical team or seek outside help to infections and death rates. State officials develop their app, officials from all nine of the noted that they included these features to selected states we interviewed said they had provide information to the public outside of limited technical expertise and resources and their agencies’ websites, which they hoped received varying levels of external support to would encourage people to download and deploy their apps, regardless of whether they use their app. Based on our observations of used customized apps or the Express option. exposure notification apps for the selected states, a common customized function was According to Google and Apple the ability to share the app with others. representatives, the Express option was Figure 6 shows screenshots for two developed to help states quickly and easily customized apps. These images illustrate how deploy their app. However, the Express an app can be tailored to offer unique option does not offer states the same functions in the user interface. For example, flexibility to tailor the functions and features one screenshot illustrates a unique function, of their app as do customized exposure “Healthcheck,” which allows app users to notification apps. report any COVID-19 related symptoms, exposure history, and testing history to their public health authority. Exposure Notification GAO-21-104622 23 Note: The number in the image on the left (490238) is an illustration of a verification code that would be provided by a public health authority to an app user to verify the positive results of a COVID-19 test. A user can then voluntarily input this code in the app to submit their recent temporary keys. Exposure Notification GAO-21-104622 24 Based on our review of the apps for the nine preparing marketing campaigns, and choosing selected states, among other qualitative to pilot the app prior to the full release. States differences between state apps, we noted that chose the Express option generally noted variation in the depth and scope of guidance shorter development times. information provided to app users. Specifically, some states provided more The cost of app development also varied detailed information on symptoms, testing, according to the information reported to us and quarantine. We also found that the apps’ by officials from the 11 states. 44 One state privacy use agreements provided varying reported zero development costs because a details on how the apps protect privacy, nonprofit organization developed the state’s including how users can delete their data. 42 app, while another state reported development costs of $700,000. 45 State officials noted that their marketing costs also 3.3 States reported that app varied; costs ranged from $380,000 to $3.2 development time and cost varied million, as of June 2021. based on several factors States used federal funding for development Officials from each of the nine selected states and marketing costs; some used state funding we interviewed and the two additional states as well. Six of the nine states reported that that provided written information varied in they used CARES Act funding to support the their reported app development time frames development of their apps or marketing costs. and costs. 43 Some public health authorities However, according to the CDC, which from these states attributed these variations distributes certain CARES Act and to several factors, including legal review and supplemental COVID-19 relief funds through marketing efforts. Officials from nine of the its Epidemiology and Laboratory Capacity for 11 states reported that the time to develop Prevention and Control of Emerging Infectious their apps ranged from less than 2 months to Diseases cooperative agreement, 46 exposure over 5 months. This time included the notification apps are allowable expenses development of the apps, as well as conducting legal reviews of contracts, through these awards, but the agency does 42 45 Some of the exposure notification apps had embedded links We did not independently verify the states’ reported costs. to the public health authorities’ websites, which provided 46 access to the state’s privacy use agreement or other As part of the Coronavirus Aid, Relief, and Economic Security information. Act (CARES), Coronavirus Preparedness and Response Supplemental Act, and Paycheck Protection Program and 43 We interviewed state public health officials from a non- Health Care Enhancement Act supplements, the cooperative generalizable sample of nine states that had deployed an agreement awarded approximately $11 billion to support the exposure notification app as of January 1, 2020: Alabama, domestic response to COVID-19. See CARES Act, Pub. L. No. Colorado, Connecticut, Minnesota, Nevada, North Carolina, 116-136, 134 Stat. 281 (2020); Coronavirus Preparedness and Pennsylvania, Virginia, and Washington. We also reached out Response Supplemental Appropriations Act, 2020, Pub. L. No. to two other states, Louisiana and Utah, which deployed apps 116-123, 134 Stat. 146 (2020); Paycheck Protection Program in the later stages of our evidence collection about the status and Health Care Enhancement Act, Pub. L. No. 116-139, 134 of their efforts to deploy an app and received written feedback Stat. 620 (2020). An additional award of $19.11 billion from the to our structured questions. Coronavirus Response and Relief Supplemental Appropriations 44 Act of 2021, Pub. L. No. 116-260, Div. M was awarded to This includes the nine selected states we interviewed and the continue to shore up domestic response efforts to COVID-19. other two states that provided written responses to our See Consolidated Appropriations Act, 2021, Div. M, Pub. L. No. questions. 116-260, 134 Stat. 1182 (2020). Exposure Notification GAO-21-104622 25 not require recipients to report on use of In addition, different levels of app use were funds to support exposure notification apps. reported by officials from the nine selected states we interviewed and the two additional 3.4 Officials reported varying states that provided written information. Specifically, for two states the number of download levels and use times their app users received exposure notifications as of June 2021 were above Different app download levels (or activations 30,000 (31,000 for one state and 42,000 for for states using the Express option) were the other), while four states reported reported by officials from the nine selected notifications that ranged from about 900 to states we interviewed and the two additional 3,800; the remaining five states did not track states that provided written information. 47 these data. However, the number of Specifically, four states reported less than 1 notifications depends on a variety of factors, million, four states reported 1 to 2 million, including the extent of the app users’ and two states reported more than 2 million contacts. downloads (or activations), as of June 2021. 48 The other state does not track these data. Further, limited data are available on the According to Google and Apple extent to which exposure notifications representatives, states that initially deployed affected people’s behavior, according to a custom app and then later added the public health officials and studies we Express option, experienced a significant reviewed. For example, public health increase in activations. Specifically, authorities do not know whether app users representatives stated that the adoption rate are actually using the app and following quadrupled for four states that added the Express option (Nevada, New Jersey, New instructions for next steps contained in the York, and Virginia). However, there may be alerts. Seven of the nine selected states that other factors that affect adoption rates. we interviewed said that they did not track Further, the number of downloads and whether app users actually sought testing or activations is not an accurate reflection of the medical care based on the receipt of an number of people using the app. For example, exposure notification from an app; one state a person could download or activate the app said it was done inconsistently and the other and not use it, or could download the app remaining state did not respond to our multiple times. See section 4.5 for additional request for these data. information on this issue and appendix II for additional information on state app adoption rates. 47 48 Download data includes Android and iOS phones in states States with customized apps can calculate download levels with customized apps; downloads for Android phones in states for Android and iOS smartphones from data obtained from using the Express option; or “activations” for iOS phones in Google and Apple apps stores. However, for states using the states using the Express (“app-less”) option. Express option, they can determine downloads for Android devices but must estimate the number of users that activated the app on iOS smartphone. Exposure Notification GAO-21-104622 26 4 Challenges Associated with Exposure Notification Apps We identified the following five categories the phone is being held or is in a pocket or of challenges associated with these apps: otherwise obstructed location, and the position of one phone with respect to the • Accuracy of measurements other phone (e.g., if it has a 90 degree • Privacy and security concerns rotation or is facing down). • Adoption and use of apps As a result, a person could receive a • Verification code delays notification even if that person was far • Evidence of effectiveness away or separated by a physical barrier from an infected person. Such a result, known as a false positive, can lead a person 4.1 Accuracy of measurements who has been notified to take unnecessary steps, such as getting tested, or self- The techniques that exposure notification quarantining. Further, false positives could apps use to measure distance have reduce that person’s confidence in the app, technical limitations that can result in users which could lead to them not using the app receiving false exposure notifications. For or using it less often. In addition, without example, BLE wireless radio technology, accurate measurements, an app could fail used to measure the distance between two to detect that two people are in close smartphones, cannot always reliably proximity for a certain amount of time, measure whether two smartphones are leaving the potentially exposed person with within 6 feet of each other. In addition, a false sense of security—a false negative. research has demonstrated that the BLE signal strength does not always decrease To help address these limitations, various with distance, and can even increase with industry experts have highlighted the distance under certain conditions. 49 For potential of using other technologies to example, objects in the environment perform measurements instead of or in between a sender and a receiver (e.g., addition to BLE, including ultra-wideband furniture, walls, people) can impact the signals and ultrasound. 50 Several studies signal, causing the received signal strength to vary substantially. Other factors include have found that these other technologies the type of phone and antenna, whether may be more accurate than BLE. 51 In 49 51 See Douglas Leith & Stephen Farrell, “Coronavirus Contact See, for example, N. Ahmed et al., “A Survey of COVID-19 Tracing: Evaluating the Potential of Using Bluetooth Contact Tracing Apps,” IEEE Access, vol. 8 (July 2020): Received Signal Strength for Proximity Detection,” ACM 134577-134601, accessed December 1, 2020, SIGCOMM Computer Communication Review, vol. 50, no.4 https://doi.org/10.1109/ACCESS.2020.3010226; and J. (2020); 1-11. Meklenburg et al., “SonicPACT: An Ultrasonic Ranging 50 Method for the Private Automated Contact Tracing (PACT) Like BLE, ultra-wideband is a wireless radio transmission Protocol,” arXiv.org (Dec. 2020): 1-14, accessed November technology, but it could provide measurements that are 24, 2020, https://arxiv.org/abs/2012.04770. more accurate. However, ultra-wideband is only available on certain newer smartphones. Ultrasound refers to the transmission of inaudible acoustic pulses in the ultrasonic frequency range between phones. Exposure Notification GAO-21-104622 27 addition, researchers have suggested that Despite the privacy protections built into exposure notification apps could use sensor the apps by Google and Apple, the public technologies to improve distance may lack confidence that their privacy is estimation based on BLE measurements, protected, in part, due to a lack of such as by using a gyroscope, an independent assessments and federal legal accelerometer, or a magnetometer. 52 For protections for the privacy of app data. In example, these technologies could help to particular, CDC’s guidance on the detect the position of the phone. However, implementation and use of exposure notification apps recommends that the thus far, there has been only limited use of apps go through independent security and these technologies. Specifically, while some privacy assessments, 53 and that the results entities with an exposure notification app be made publicly available. 54 However, we use ultrasound technology for distance found that none of the nine selected states estimation, including several U.S. had fully implemented this guidance. universities, as of June 2021, we did not Specifically, officials from five of the nine find any exposure notification apps that use selected states reported that security and ultra-wideband signals, a gyroscope, privacy assessments were performed; accelerometer, or a magnetometer. however, the results were not made publicly available. The remaining four states 4.2 Privacy and security concerns reported that these assessments were not performed. Privacy Currently there is no federal law that Officials from all nine of the selected states provides the public with clearly applicable identified privacy as an important factor in privacy protections for the information that determining whether to implement an exposure notification apps gather. exposure notification app and in selecting Specifically, in January 2019, we reported the system used by the app (i.e., Google that the U.S. did not have a comprehensive Apple system). In particular, officials stated internet privacy law governing the that users would likely not adopt an app collection, use, and sale or other disclosure that collected their personal information, of consumers’ personal information. including location data. Accordingly, we recommended that Congress consider developing legislation on 52 document detailing the process and the outcome of the A gyroscope is a device used for measuring or maintaining orientation and angular velocity. An accelerometer is a analysis. See Office of Management and Budget, Managing device used to measure acceleration forces. A Information as a Strategic Resource, Circular A-130 magnetometer is a device that measures the strength and (Washington, D.C.: July 2016). sometimes the direction of magnetic fields. 54 Centers for Disease Control and Prevention, Guidelines for 53 the Implementation and Use of Digital Tools to Augment An example of such an assessment is a privacy impact assessment that is used by federal agencies in response to Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec. requirements in the E-Government Act of 2002. Among 15, 2020) and Preliminary Criteria for the Evaluation of other things, the assessment is an analysis of how personally Digital Contact Tracing Tools for COVID-19, version 1.2 identifiable information is handled to ensure compliance (Atlanta, Ga.: May 17, 2020). with applicable privacy requirements and manage privacy risks. Also, a privacy impact assessment includes a formal Exposure Notification GAO-21-104622 28 internet privacy that, among other things, Security would enhance consumer protections. 55 Legislation governing the collection and use To ensure that exposure notification apps of consumers’ personal information—in function as intended and that user privacy particular for exposure notification apps— is protected, it is important that developers could help to safeguard their privacy, and build in security protections. However, provide the public with greater assurance security assessments of these apps are that its privacy is protected. However, such limited. legislation has not been enacted. 56 Security considerations should include the In the absence of such legislation, individual supporting infrastructure—such as central companies have set their own privacy servers—and address how the data are requirements for exposure notification stored and maintained, including apps, including requirements on the appropriate authentication and access collection and use of the data. For example, controls. Security incidents could lead to Google and Apple have each established privacy violations (e.g., identifying or requirements for their exposure notification tracking users) or disrupt the functioning of system and their respective app stores regarding data collection and privacy. These the app (e.g., inserting false data). This requirements specify that only the would likely result in the public’s loss of minimum amount of user data that is confidence in the apps, potentially leading necessary for response efforts should be to decreased use. Researchers have collected, and that the data may only be identified a variety of potential threats for used for such efforts. 57 In addition, the requirements state that the apps cannot collect any information to identify or track the precise location of users. 55 specifically address exposure notification apps or the GAO, Internet Privacy: Additional Federal Authority Could Enhance Consumer Protection and Provide Flexibility, associated privacy issues. In addition, at least one state GAO-19-52 (Washington, D.C.: Jan. 15, 2019). Other federal passed a law regarding the use of location data for contact laws governing health information, including the Health tracing. Specifically, in June 2020 Kansas passed a law stating Insurance Portability and Accountability Act, may not that contact tracing shall not be conducted through the use provide consistent, clearly-applicable privacy protections for of any service or means that uses cell phone location data to the information that likely would be gathered and used in identify or track, directly or indirectly, the movement of digital contact tracing activities. See Congressional Research persons. See K.S.A. § 48-961 (2021). Service, COVID-19: Digital Contact Tracing and Privacy Law, 57 See, for example, Google, Google COVID-19 Exposure LSB10511 (Washington, D.C.: July 9, 2020). Notifications Service Additional Terms, (last modified May 4, 56 2020), accessed May 16, 2021, Congress has introduced several bills over its last two sessions that address aspects of exposure notification apps https://blog.google/documents/72/Exposure_Notifications_ or digital contact tracing tools. Of the bills that have been Service_Additional_Terms.pdf; and Apple, Exposure introduced, one that was enacted into law related to Notification APIs Addendum, (last revised May 4, 2020), implementing a national strategy for contact tracing and accessed May 16, 2021, enhancing information technology and data modernization https://developer.apple.com/contact/request/download/Ex capabilities (American Rescue Plan Act of 2021, Pub. L. No. posure_Notification_Addendum.pdf. 117-2, § 2401, 135 Stat. 4, 40 (2021)). However, it does not Exposure Notification GAO-21-104622 29 exposure notification apps. 58 Table 3 identifies several examples of these threats and their potential effects. Table 3: Examples of threats and their effects for exposure notification apps Threat Description Potential Effect Re- Comparing exposure notifications with The identity of an infected app user is revealed. identification personal logs of a phone owner’s recent contacts. Denial of Broadcasting fake encounter messages to Loss of availability of a smartphone due to the service consume resources of other smartphones. additional battery power, storage, and processing time required to store and process the fake messages. Phone Tracking a user’s location by analyzing the A user’s location and movements is revealed. tracking information sent in encounter messages, such as the random identifiers. Relay (or Re-transmitting captured encounter Smartphone receives exposure notification despite replay) messages at the same or a different not coming in close contact with an infected person location. (i.e., false positive). Source: GAO review of selected literature. | GAO-21-104622 The Google Apple system includes features apps are a relatively new technology, these intended to mitigate these threats. In assessments have, as of now, limited data addition, according to representatives from and results. Further, as previously stated, these companies, they had a third party the selected states have not provided the perform a security assessment of the results of independent security assessments system. There are also ongoing assessments in a public format, as recommended by CDC on the security of exposure notification guidance. 60 apps, including on the Google Apple system. 59 However, as exposure notification 58 testing and validation services. In April 2021, the company See, for example, N. Ahmed et al., “A Survey of COVID-19 Contact Tracing Apps”; Massachusetts Institute of reported that it had identified a vulnerability with apps using Technology Lincoln Laboratory, “Exposure Notification the Google Apple system. Specifically, the company reported Security Assessment Considerations,” Lexington, that preinstalled apps could gain access to system logs made Massachusetts. Unpublished Article; and M. Chowdhury et by exposure notification apps on Android devices. According al., “COVID-19 Contact Tracing: Challenges and Future to the company, these logs could include information, such Directions,” IEEE Access, vol. 8 (Nov. 2020): 225703-225729, as whether a person had received an exposure notification accessed February 16, 2021, and the random identifiers that a smartphone device had https://ieeexplore.ieee.org/document/9252092. sent and received. Google representatives stated that a fix for this vulnerability was available as of May 5, 2021, and 59 For example, the Massachusetts Institute of Technology’s that there is no evidence that it was exploited. Lincoln Laboratory has developed security assessment 60 Centers for Disease Control and Prevention, Guidelines for considerations for the Google Apple system and conducted a security assessment of one of the apps used in the states. In the Implementation and Use of Digital Tools to Augment addition, in February 2021, DHS’s Science and Technology Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec. Directorate’s Silicon Valley Innovation Program awarded 15, 2020). funding to a company (AppCensus) for a project to develop Exposure Notification GAO-21-104622 30 4.3 Adoption and use of apps users, which followed other incidents of misuse of personal information. 62 Also, States have also faced challenges attracting multiple officials noted that the public was public interest in downloading (or skeptical of the Google Apple system, since activating) and using an exposure app. State it is a joint initiative between U.S. public health officials told us that, in spite technology companies and the government of their marketing and outreach efforts, that involves personal health information. getting people to download (or activate) In particular, officials from three states and use their app is difficult for the indicated that the public expressed following reasons. concerns about the perceived “big brother” nature of exposure notification. Lack of trust. Mistrust of governmental health authorities and technology Lack of understanding of how apps companies can lead people to forgo using function. Multiple officials said they apps, according to literature and state frequently had to counter misinformation officials. For example, officials from six of about how the apps work and the data they the 11 states cited public concerns about collect. For instance, officials from one state the use of apps for government surveillance reported that they emphasized the app's (e.g., using the apps to track users’ location) use as a public health communication tool as a leading obstacle to app adoption, 61 because of misinformation describing the even though these apps do not collect exposure notification app as a data location data. In addition, the public may collection tool used to surveil and track the not trust big technology companies with public. their data. These concerns may be exacerbated by reported vulnerabilities Also, multiple officials said the public had a with apps using the Google Apple system. limited understanding about the apps’ Specifically, as previously stated, a company privacy-preserving features. Officials from reported that other apps on a phone could three states said that they believed the potentially gain access to sensitive public did not understand the technical information, including whether a person aspects of the apps, which may include how had received an exposure notification. In the random identifiers do not reveal addition, the lack of trust regarding the use personal information. 63 Such of apps may be intensified by other misunderstandings may contribute to public incidents where technology companies unwillingness to download or use the apps. potentially misused consumers’ personal Further, such misunderstandings may also information. For instance, in April 2018, contribute to hesitance to enter verification Facebook disclosed that a Cambridge codes for people who had downloaded an University researcher may have improperly app on their device. Specifically, sometimes shared the personal data of 87 million of its people receive positive COVID-19 test 61 63 The 11 states include nine from our selected sample plus According to Google and Apple documentation, the two additional states. random identifiers exchanged with other smartphones are 62 not linked to the app user’s identity or phone number and GAO-19-52. change on a periodic basis (e.g., every 10 minutes). Exposure Notification GAO-21-104622 31 results long after the app has been reason they chose not to deploy an app was downloaded. Because people may not have the lack of necessary supporting initially understood (or have forgotten) how infrastructure or internet service in rural the app works, including the apps’ built-in areas. To be effective, exposure notification privacy preserving features, some app users apps need to be downloaded and used by a many not want to input their verification critical mass of the general public. While codes to prompt exposure notifications to the levels of adoption needed to achieve other people, according to officials from certain measures of effectiveness are not selected states. well established (see section 4.5), increasing the number of people using the app should Lack of awareness of the availability of the result in a greater likelihood that users who apps. Officials from several states noted come in close contact with an infectious that it was difficult to make people aware of person will be notified of potential the apps. For example, officials from exposure, according to CDC and the multiple states noted that they thought Massachusetts Institute of Technology’s building awareness in closely connected Lincoln Laboratory. communities with influential leaders would encourage people to download and use an app. However, one state tried, but was 4.4 Verification code delays unsuccessful, in recruiting support from some of these groups, including churches States were challenged in distributing and a college football program. One official verification codes quickly to the public. As said the lack of support was a lost previously stated, these codes are used to opportunity to build awareness and confirm that a person had a positive test or increase that state’s app adoption rate. diagnosis before they are able to upload Further, some states reported having their recent temporary keys to the National minimal resources for marketing, which one Key Server. For people to be notified of official said resulted in low awareness of potential exposure quickly, these the state’s app. Three states reported that a verification codes need to be distributed in federally led national marketing campaign a timely manner and users need to would have helped promote their app and voluntarily decide to use them to notify drive higher rates of adoption. Similarly, recent contacts. officials from a national health organization reported that a national public awareness Officials from several of the selected states campaign led by the CDC would help reported that their initial process for encourage adoption and be more cost- distributing the codes required a public effective than individual state campaigns. health official, such as a contact tracer, to provide a person with the verification code Limited access. Another reason it can be via phone after the person had received a difficult to get people to download and use positive test result or a confirmed diagnosis. an exposure notifications app is lack of However, states reported that, due to access to a smartphone, reliable cellular staffing shortages, in particular as cases coverage, and broadband internet service, surged, it sometimes took several days to according to selected states and literature. provide the code. As a result, some app Indeed, officials in a few states told us one users who had tested positive for COVID-19 Exposure Notification GAO-21-104622 32 were delayed in submitting their website with instructions for how to obtain verification codes to notify others of the code. State officials reported that this possible exposure, according to officials automated distribution resulted in an from selected states. 64 In addition, there increase in the number of verification codes can be delays in test results being available disseminated to app users. For example, and provided to health care providers who following implementation of the new report the results to local or state health process, one state's average distribution of officials. In particular, earlier in the verification codes increased from 15 to 85 a pandemic, testing availability and day, according to public health officials. In turnaround time for results could take a addition, officials from a different state week or more. Following the receipt of test reported that they had seen an increase in results, the health care provider or the number of codes redeemed, and laboratory then reports the results to local improved the timeliness of code or state health officials. Any delays in this redemption following implementation of process could also contribute to delays in the new system. However, another state public health authorities’ distribution of verification codes to individuals after the noted that, even after automating the individual has received the confirmed process, it still took 4 days, on average, for diagnosis. a person to receive a verification code following a positive test result. In addition, a few states noted that contact tracers did not always follow the state’s 4.5 Limited evidence of processes for providing app users with a effectiveness verification code to enter COVID-19 test results in the app. For example, when a We found limited evidence that exposure contact tracer was conducting an interview notification apps are effective at enhancing with a person, they were supposed to ask if the speed or reach of manual contact the person had downloaded the state’s app, tracing or at reducing the spread of disease. and if so, to provide them with a One reason for the dearth of evidence is verification code. However, officials from that states collect limited data from one state stated that this was not always exposure notification apps due to the performed. emphasis on data privacy. In addition, little to no guidance exists on what data to To help address this challenge, five of the collect and how to collect the data. As for nine selected states implemented an slowing the spread of COVID-19, studies automated process to distribute the codes. have not yet sufficiently demonstrated that Instead of providing the codes entirely exposure notification apps are having an through phone calls, some states also send effect. CDC and others have, therefore, text messages with the code or a link to a 64 If the user tests positive for COVID-19, a public health authority generates a verification code and then sends the user the code to verify the positive test results. A user can then voluntarily input this code in the app to submit recent temporary keys. Exposure Notification GAO-21-104622 33 highlighted the need for additional research diseases like COVID-19, the speed of into the effectiveness of exposure contact tracing plays a critical role in notification apps in preventing the spread reducing disease spread, allowing those of disease. who may have been exposed to take action more quickly. States could use the speed of 4.5.1 States collect limited data from notification as one indicator or metric of exposure notification apps app effectiveness. According to Google and Apple representatives, the time between The privacy protections that are when a user submits their temporary keys incorporated into the functionality of the to the key server and when a person would existing apps limit the data available to be notified is estimated to be between 4 public health authorities, which reduces the and 10 hours. However, a few factors can ability to measure and improve the delay the submission, including the amount effectiveness of the apps. For example, of time it takes to receive a test result (e.g., officials from all nine selected states said to time for test processing and reporting to preserve personal privacy, they do not the health care provider and to the local collect data on who has installed an app, health officials) and when that person including who has received an exposure receives and uses the verification code. notification. In addition, states have limited data on how well the apps are working, 4.5.2 States lack guidance on measuring including how changes to the formula used effectiveness to calculate the level of risk affects the number of people provided with exposure notifications. 65 In addition, states do not States lack guidance for measuring the effectiveness of exposure notification apps. collect location data, so they are unable to Officials from nearly all the selected states identify where disease spread is occurring. told us they wanted to gauge the impact of their apps and assess effectiveness, such as Furthermore, states do not collect data on the enhanced reach through electronic the speed of exposure notification, notification. Officials from several of the according to our review of information selected states said that they had reached provided by selected states. Metrics on the out to CDC for guidance regarding speed of notification are not provided as recommended approaches and indicators part of the Google Apple system, though for measuring app effectiveness, which was these data could be collected by the states confirmed by CDC officials. However, the individually, should they choose to, requested information was not available. according to Google and Apple CDC officials indicated that they considered representatives. With fast-spreading developing additional guidance to evaluate 65 MITRE is planning to enhance the Exposure Notification Private Analytics portal with additional features, including the ability to analyze how changes in risk parameters affect the number of exposure notifications. The portal and the exposure notification analytics data it provides are only available to states using the Express option. Exposure Notification GAO-21-104622 34 app effectiveness. However, they or reach compared to manual contact acknowledged there are limited app tracing. 67 evaluation strategies available due to the lack of data from exposure notification apps The lack of standardized metrics was and, as a result, they are not planning to identified as a challenge in President develop additional guidance. Because of the Biden’s National Strategy for the COVID-19 lack of federal guidance, officials from Response and Pandemic Preparedness in states said they were “on their own” and January 2021. The strategy noted that began reaching out to other states and states use and report different metrics for countries that had deployed apps for advice tracking COVID-19 response activities, and best practices, such as metrics for including contact tracing, and called for measuring effectiveness. Officials from the common federal metrics to evaluate majority of the selected states said they progress and the identification of areas wished there had been additional guidance where additional federal resources should available, including how to measure app be directed. 68 effectiveness; officials from three states used the analogy of “building the plane In part due to the lack of federal guidance, while flying it,” to describe their experience selected states varied in the types of data deploying their apps with limited direction. that they are collecting to measure the overall effectiveness of their own apps and CDC has developed general guidance on have developed their own metrics and exposure notification apps, such as indicators for determining how well the minimum and preferred characteristics. 66 apps are working. States are using one or CDC also developed guidance to measure more of these metrics: the success of manual contact tracing efforts, including both process and outcome • App downloads or activations 69 metrics, but has not developed specific • Verification codes issued 70 guidance on criteria to use in measuring app effectiveness, such as increased speed • Verification codes claimed 71 66 68 Centers for Disease Control and Prevention, Preliminary The White House, National Strategy for the COVID-19 Criteria for the Evaluation of Digital Contact Tracing Tools Response and Pandemic Preparedness, (Washington, D.C., for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and Jan. 21, 2021). Guidelines for the Implementation and Use of Digital Tools to 69 Augment Traditional Contact Tracing, version 1.0 (Atlanta, As previously mentioned, download data includes Android Ga.: Dec. 15, 2020). CDC’s May 2020 and December 2020 and iOS phones in states with customized apps; downloads guidance and website information on digital contact tracing for Android phones in states using the Express option; or tools did not include a definition for effectiveness or any “activations” for iOS phones in states using the Express standardized metrics for states or indicators for measuring (“app-less”) option. app effectiveness. 70 Verification codes issued refers to the codes that are 67 disseminated to app users with a positive COVID-19 test Centers for Disease Control and Prevention, Evaluating Case Investigation and Contact Tracing Success, (Atlanta, result. The codes may be provided on the phone by contact Ga.: May 26, 2020), accessed June 9, 2021, tracers or through other methods, such as text message. https://www.cdc.gov/coronavirus/2019-ncov/php/contact-t 71 Verification codes claimed refers to the codes entered by racing/contact-tracing-plan/evaluating-success.html. app users with a positive COVID-19 test result that enables them to send the recent temporary keys to the National Key Server to notify others that they may be at risk. Exposure Notification GAO-21-104622 35 • Exposure notifications generated 72 regard to reach, the number of downloads gives an approximate, but not actual sense Yet, officials from eight of the nine selected of the number of app users. states noted that some of these metrics provide a limited understanding of app The number of downloads is not an effectiveness. Data on downloads, accurate reflection of app usage. Officials verification codes, and exposure from one state speculated that some notifications provide public health people were downloading the apps out of authorities some information to gauge how curiosity but never enabling them on their effective they are. However, these metrics smartphone. Further, after a person do not indicate how quickly people were downloads the app, they need to perform a notified of exposure or timeliness relative series of actions for it to be used as to manual contact tracing alone. With intended (see fig. 7). Therefore, a user 72 Exposure notifications are alerts provided to close contacts of the app users who confirm a positive test or diagnosis using a verification code they enter into the app. However, they are only an estimate, as app users voluntarily provide this information. Exposure Notification GAO-21-104622 36 could download but not use the app or according to officials from one state. download it more than once, according to Further, because states do not track who public health officials from several selected receives an exposure notification, states states. Also, people could choose to not have a limited understanding of what enable receiving exposure notifications; impact, if any, these notifications have on ignore notifications; and if they do test disease spread and the overall effectiveness positive, not voluntarily provide that of their apps. information. While download totals may represent the possible population of app Nonetheless, public health officials from users that could receive benefits (i.e., seven of the nine selected states said they notification) from these apps, these believe the exposure notification apps have limitations hinder states from been effective and that their apps had been understanding the effectiveness of using worthwhile. Officials from two states said exposure notification apps. they think adoption even at relatively low levels would help slow disease spread. Verification codes claimed by the people Furthermore, exposure notification apps who have tested positive for COVID-19 and provided a new tool for states to use—at a who entered the information in the app time of urgent need—to limit the spread of (which prompts exposure notifications to be COVID-19. sent to others) may also provide an indication of app use. Similarly, the number 4.5.3 Evidence of reduced disease of exposure notifications gives a sense of spread has been limited but additional how many people are notified of potential studies are underway exposure to COVID-19, but it may provide limited insight into the effectiveness of the We reviewed seven selected modeling public health intervention because users studies that have sought to measure the need to voluntarily provide this information effects of the use of apps on the spread of in their apps. Finally, none of the data COVID-19. However, there are important indicate whether people changed behavior limitations to these studies—such as limited as a result of the notification. evidence to support assumptions about behavioral changes—which hinder the While people seeking testing or medical ability to draw high-confidence conclusions care could be asked whether they sought about the apps effectiveness. 73 The studies testing or care due to an exposure we reviewed generally suggested that the notification from an app, such information use of exposure notification apps can may violate a user's expectations of privacy, reduce disease transmission. 74 In general, 73 74 Infectious disease models are simplified versions of reality The selected studies we reviewed covered a range of that help to characterize disease spread (see GAO-20-372 geographic areas, including Washington State, the United for an overview of infectious disease modeling). Other types Kingdom, Spain, and Switzerland, and were published of epidemiological studies of contact tracing apps that could between April 2020 and May 2021. Five of the seven papers be conducted in real world settings, rather than via are peer-reviewed publications. We identified the papers modeling, face methodological, logistical, and ethical from our interviews with subject matter experts and a challenges, including the lack of empirical data, confounding search of the literature. Studies we reviewed include: factors that affect disease spread, and other issues. Exposure Notification GAO-21-104622 37 the studies suggested that app usage can isolated. In some cases, particularly with decrease COVID-19 infections and deaths, studies earlier in the pandemic, these with the size of the estimated effects assumptions were not grounded in research depending on the level of app adoption, and were not otherwise well supported. For among other things. For example, one peer- example: reviewed study estimated that, when 15 percent of the population used an exposure • Assumptions in one study were that notification app, infections could be everyone notified of a potential reduced by approximately 8 percent and exposure would self-isolate, with a 2 deaths by about 6 percent. Another peer- percent drop-out rate each day, and reviewed study in the United Kingdom that 18 percent of infected people estimated that a 30 percent app uptake remained asymptomatic, with no averted approximately one infection for variation in this rate across age every four infections that arose over a 4½- groups. 75 These assumptions were not month period. grounded in evidence because little to none was available at the time. However, there are significant limitations to these modeling studies. For example, the • A study of three counties in Washington models estimated outcomes by relying on State assumed in its simulations that it assumptions about app usage and would take 2 days from symptom onset behavioral changes associated with to receive a COVID-19 test result, which notifications. These assumptions covered the authors characterized as a key factors such as how many people used an assumption underlying the findings. app, how many app users had a positive However, in the earlier months of the test result, and how many app users self- R. Hinch, et al., Effective Configurations of a Digital Contact D. Menges, et al., “A Data-Driven Simulation of the Exposure Tracing App: A Report to NHSX, (April 16, 2020), accessed Notification Cascade for Digital Contact Tracing of SARS-CoV- December 9, 2021, 2 in Zurich, Switzerland,” JAMA Network Open, (4 https://cdn.theconversation.com/static_files/files/1009/Rep (4):e218184), (April 30, 2021), accessed July 13, 2021, ort_-_Effective_App_Configurations.pdf?1587531217. https://jamanetwork.com/journals/jamanetworkopen/fullar M. Abueg, et al., “Modeling the Effect of Exposure ticle/2779376. Notification and Non-pharmaceutical Interventions on Massachusetts Institute of Technology Lincoln Laboratory, COVID-19 Transmission in Washington State,” npj Digital “Simulated Automatic Exposure Notification (SimAEN): Medicine, (4, 49), (March 12, 2021) accessed March 12, Exploring the Effects of Interventions on the Spread of 2021, https://www.nature.com/articles/s41746-021-00422- COVID,” Private Automated Contact Tracing (PACT) 7. Technical Report #3, (December 8, 2020), accessed March 1, C. Wymant, et al., “The Epidemiological Impact of the NHS 2021, https://pact.mit.edu/simulated-automatic-exposure- COVID-19 App,” Nature, Vol. 594, no. 7863 (2021).pp. 408- notification-simaen-exploring-the-effects-of-interventions- 412, accessed February 25, 2021. on-the-spread-of-covid-wlogos/. 75 P. Rodríguez, et al., “A Population-Based Controlled Estimates of the COVID-19 asymptomatic rates vary Experiment Assessing the Epidemiological Impact of Digital widely by age group, according to information from CDC. Contact Tracing,” Nature Communications, (January 26, Centers for Disease Control and Prevention, “Estimated 2021), accessed February 22, 2021, Disease Burden of COVID-19,” COVID-19, (Atlanta, Ga.: https://www.nature.com/articles/s41467-020-20817-6. updated May 19, 2021), accessed July 13, 2021, S. Marcel, et al., “Early Evidence of Effectiveness of Digital https://www.cdc.gov/coronavirus/2019-ncov/cases-updates Contact Tracing for SARS-CoV-2 in Switzerland,” Swiss /burden.html. Medical Weekly, (December 16, 2020), accessed March 9, 2021, https://smw.ch/article/doi/smw.2020.20457. Exposure Notification GAO-21-104622 38 pandemic, wait times for test results in underway that suggest the use of apps can U.S. could be a week or more. 76 help mitigate the spread of COVID-19. 77 In addition, some states are conducting their • Oxford University researchers own evaluations of the effectiveness of associated with the United Kingdom exposure notification apps in reducing studies told us that the recent rise in disease spread. variant strains and vaccinations has increased uncertainty in assumptions about disease transmission. 4.5.4 CDC and others have highlighted the need for additional research and In addition to the studies on disease spread, data some studies have estimated shorter-term outcomes, such as the number of close Exposure notification apps are a relatively contacts detected by exposure notification recent public health intervention. As a apps. In one simulation study, the findings result, additional primary research on the implied that the app prompted quarantine benefits and effectiveness of exposure notification apps is needed, according to recommendations for, at most, an CDC and other public health researchers. 78 estimated 5 percent more exposed contacts This includes a need for primary research than manual contact tracing. However, as into the use of digital tools in conjunction with the modeling studies we reviewed on with manual systems, since public health disease spread, these studies of shorter- authorities are unlikely to use digital tools term outcomes are also subject to in isolation, according to all selected states important limitations, such as model inputs and most literature we reviewed. derived from studies with limited sample Specifically: sizes or national estimates applied to states or local regions. • CDC has noted that more data are needed from preliminary Since we originally identified papers for our implementation efforts to quantify the review, additional studies are now public health value of these apps. 79 76 78 D. Lazer, et al., “Report #8: Failing the Test: Waiting Times A. Anglemyer, et al., “Digital contact tracing technologies for COVID Diagnostic Tests Across the U.S.” in The State of in epidemics: a rapid review.” Cochrane Database of the Nation: A 50-State COVID-19 Survey, (OSF Preprints, Systematic Reviews. (2020). August 2020), accessed July 13, 2021, 79 https://doi.org/10.31219/osf.io/gj9x8. Centers for Disease Control and Prevention, Guidelines for the Implementation and Use of Digital Tools to Augment 77 See, for example, C. Segal, et al., Early Epidemiological Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec. Evidence of Public Health Value of WA Notify, a Smartphone- 15, 2020). based Exposure Notification Tool: Modeling COVID-19 Cases Averted in Washington State (June 2021), accessed July 1, 2021, https://www.medrxiv.org/content/10.1101/2021.06.04.212 57951v4; and J. Masel, et al., Quantifying meaningful adoption of a SARS-CoV-2 exposure notification app at the campus of the University of Arizona (June 2021), accessed June 1, 2021, https://www.medrxiv.org/content/10.1101/2021.02.02.212 51022v6. Exposure Notification GAO-21-104622 39 Also, the agency has highlighted the effectiveness of exposure notification need for more studies on the apps is needed, since there is currently effectiveness of digital tools, including a limited understanding of the extent to exposure notification apps, to support which apps may have changed the contact tracing and reduce the spread course of the pandemic in the U.S. 81 of infectious disease. In addition, CDC The organization called for a data has identified a specific research need driven approach and research on app to comprehensively compare the efficacy to help app developers and effectiveness of manual contact tracing others, including the federal with exposure notification apps and has government, decide whether to initiated work to study these issues with improve apps for potential future use or the Massachusetts Institute of abandon the approach if research Technology’s Lincoln Laboratory, showed that desired outcomes had not according to CDC officials. been achieved. • WHO has called for additional research on the minimum adoption levels Moreover, the need for additional research required for these apps to be effective on the effectiveness was identified as a in light of the limited evidence to challenge in President Biden’s National date. 80 In addition, since some Strategy, issued in January 2021. The populations have limited access to strategy notes that the federal government digital technology, WHO identified the should work with public health authorities potential for the systematic exclusion of and the private sector to collect COVID-19 individuals who cannot access such data on a range of issues, including the effectiveness of contact tracing. 82 technologies. It called for additional research and sufficient regulatory oversight of these issues. • Linux Foundation Public Health noted that additional research on the 82 The White House, National Strategy for the COVID-19 80 World Health Organization, Contact Tracing in the Context Response and Pandemic Preparedness, (Washington, D.C., of COVID-19, Interim Guidance, February 1, 2021. Jan. 21, 2021). 81 Linux Foundation Public Health was founded in summer 2020 with an initial focus on helping public health authorities deploy apps based on the Google Apple system. Exposure Notification GAO-21-104622 40 5 Policy Options That Could Help Address Challenges for Future Use We identified four policy options that, when result in users receiving false exposure implemented, could help address the notifications, such as by improving the challenges we have identified for both accuracy of the distance measurements current and future use of exposure performed by exposure notification apps. notification apps. 83 Policymakers could also For example, apps could use additional choose to maintain the status quo—that is, sensors (e.g., the gyroscope and allow current efforts to proceed without magnetometer that certain smartphones intervention. The relevant policymakers already have) or other technologies, such as ultra-wideband and ultrasound. In addition, could include Congress, other elected research could examine methods for officials, federal agencies, state and local evaluating other factors that affect the risk governments, academic research of disease transmission, such as whether institutions, and industry. While some the encounter occurred indoors or challenges described in this report may be outdoors. addressed through current efforts, other challenges may not be resolved, may be Policymakers could promote research in exacerbated, or may take longer to resolve multiple ways, including by providing grants without intervention. The four policy to academic and research institutions or by options are in the following areas: research setting up a public-private partnership. and development, privacy and security, Further, the research could build off of prior data collection and measurement, and and ongoing research by various entities. national strategy. Opportunities 5.1 Policy option: Research and • Research on technological limitations, development such as inaccurate distance measurements, could help increase the Policymakers could promote research and accuracy and speed of exposure development to address technological notification apps, incentivizing users to limitations. download and use them. • Research on technologies and Description architectures other than those used by U.S. states could also improve the apps, Research could seek to address the technical limitations we identified that can for example by increasing the speed 83 that would be needed to fully implement a specific policy We present policy options that were within the scope of this technology assessment. This is not an exhaustive list of option or combination of options—for instance, on potential all potential policy options, nor are policy options intended design and legal issues—nor did we assess how effective the to be recommendations to federal agencies or matters for options may be. We express no view regarding the extent to congressional consideration. They are not listed in a specific which legal changes would be necessary to implement them. rank or order, and we are not suggesting that they be completed individually or combined in any particular fashion. We did not conduct the detailed additional analysis Exposure Notification GAO-21-104622 41 and reach of notifications. Such smartphone technology would require alternatives include GPS and centralized ongoing research. or hybrid data architecture. • Partnerships with technology 5.2 Policy option: Privacy and companies could help with integrating security standards and best improvements into smartphone practices operating systems. These collaborations could spur further technological Policymakers could promote uniform innovation. privacy and security standards and best Considerations practices for exposure notification apps. • Research into new technologies could Description be costly and is generally considered a long-term investment with uncertain Policymakers could support the benefits. development of privacy and security • The roles for government, the private standards and best practices for exposure sector, and academia in researching notification apps to ensure that these apps new technologies for exposure function as intended and that user privacy notification apps would need to be is protected. One way to do this would be defined, planned, and coordinated to to specify standards for public health ensure that research and costs are not authorities to ensure that personal data are duplicative. encrypted when stored, and to specify limits on the types of data that can be • Research may not produce cost- collected and how the data may be used effective improvements, because and disclosed. In addition, the standards existing apps may still be sufficiently could specify that the data can only be used accurate for notifying a person of for disease response efforts and that potential exposure. Moreover, other personal data cannot be shared with other alternative technologies also have agencies, law enforcement, or immigration accuracy limitations, and other data authorities without a user’s consent. architectures may increase the risk of revealing sensitive user information. Another action policymakers could take is to require that standards be developed and • Research into new technologies based agreed on by a broad coalition of on the COVID-19 pandemic may also stakeholders. Best practices could also be result in apps that are not functional for developed by government agencies (e.g., future outbreaks or pandemics. NIST) or the private sector. Diseases that are not transmitted through the air, such as sexually Opportunities transmitted diseases, would require apps that use different methods to • Developing and adopting uniform determine potential exposure. In privacy and security standards and addition, the continuous changes in related best practices could help address real and perceived risks that Exposure Notification GAO-21-104622 42 the public’s data might be misused or Description otherwise not appropriately protected. Policymakers could assess the approaches • Standards developed and agreed on by that states have used to increase adoption a broad coalition of stakeholders could and then develop best practices based on increase the likelihood of still broader those results. Best practices could also stakeholder agreement and buy-in. include standardization of the metrics • Independent security and privacy collected and reported to measure assessments could evaluate apps based effectiveness as well as the procedures for on these standards and best practices, verification code distribution. The and these assessments could be made development of best practices could be led publicly available. by a broad coalition of stakeholders and result in guidance to states. These efforts Considerations could help address the challenges we • Policymakers would need to balance identified related to app adoption, the need for privacy and security with verification code delays, and efficacy the direct and indirect costs of determination. developing and implementing these standards and practices. Opportunities • Implementing these privacy • Best practices could help state public requirements may require flexibility health authorities share strategies to because different jurisdictions could improve app adoption. For example, if a use different technologies (e.g., BLE or state found that translating the app into GPS) and data architectures to collect multiple languages improved adoption and use the data. among non-English speaking people, • It could be challenging to determine this information could be shared with how to oversee and enforce the privacy other states. Also, understanding and and security standards and practices. appealing to user motivations could promote app adoption. Further, 5.3 Policy option: Best practices to partnering with trusted sponsors could measure effectiveness encourage cooperation in COVID-19 contact tracing, as published by the Policymakers could promote best practices National Academies of Sciences, to increase adoption and measure the Engineering, and Medicine. 84 effectiveness of exposure notification apps. • Such practices could help state public health authorities by providing information on potential methods and 84The National Academies of Sciences, Engineering, and Medicine, Encouraging Participation and Cooperation in Contact Tracing: Lessons from Survey Research, (Washington, D.C.: Aug. 2020). Exposure Notification GAO-21-104622 43 processes for distributing verification deployment, and use of exposure codes in a timely manner. notification apps. • In addition, best practices can help states to measure the effectiveness and Description impact of these apps. Best practices could also leverage outside knowledge Policymakers could evaluate whether to to promote app adoption. More enhance the current national strategy or a future pandemic response strategy to accurate measurement of app enable a coordinated nationwide approach effectiveness would help public health to the development and deployment of authorities identify opportunities for exposure notification apps. 85 This could improvement, both to the technology’s help address the challenges we identified function and to its widespread use. related to the adoption of these apps and evidence of their effectiveness. An Considerations enhanced strategy could include specifying federal, state, and local roles and • The creation of best practices could coordination efforts. Further, an enhanced require consensus from many public strategy could identify what other and private sector stakeholders, which infectious diseases (e.g., tuberculosis, can be time- and resource-intensive. measles) may be applicable to exposure notification apps in the future. • If the best practices are not updated, they may not be relevant or useful in a As part of this strategy, the federal future pandemic. government could decide to repurpose • In some cases, stakeholders may lack apps that were developed for state use sufficient or complete information or during the COVID-19 pandemic, or use the experience to develop best comparable technology to develop new practices. If best practices are put in contact tracing solutions. Policymakers place without sufficient basis, it could could recommend a national exposure limit further innovation. notification app that public health authorities could decide to use based on 5.4 Policy option: Enhance the their individual needs, resulting in a generic exposure notification app tailored to state national strategy needs or a federally managed exposure notification app that is made available for Policymakers could collaborate to enhance states to use. the national strategy and promote a coordinated approach to the development, 85 The White House, National Strategy for the COVID-19 Response and Pandemic Preparedness, (Washington, D.C., Jan. 21, 2021). Exposure Notification GAO-21-104622 44 Opportunities Considerations • Enhanced national coordination could • Implementing a coordinated national prompt faster deployment of apps in strategy would likely have associated the future if that coordination builds costs and require a source of sustained upon the underlying infrastructure and funding during and after the pandemic. leverages the lessons learned from • Without clear roles and responsibilities, COVID-19. coordination culd be challenging. For • A federally led national marketing example, coordination of groups with campaign with cohesive and coherent divergent perspectives and interests messaging could result in wider may pose challenges to defining adoption of exposure notification apps. outcomes and measuring performance Increased federal promotion and and effectiveness of apps. support of the exposure notification • It is unclear whether the public would apps could potentially help with be more or les likely to trust and use a increasing public trust in the apps. national exposre notification app than • A national app could allow integration one developed by their state of exposure notification capabilities government. Some states, including with other disease prevention and Virginia, Colorado, and California, have response activities, such as test passed state-wide privacy laws. Due to scheduling or vaccine delivery the absence of a federal privacy law in coordination. the U.S., the public may be less likely to trust the federal government’s privacy protections. Exposure Notification GAO-21-104622 45 6 Agency and Expert Comments We provided a draft of this report to the Departments of Health and Human Services (including CDC and NIH), Homeland Security, and Commerce, Federal Communications Commission, and Federal Trade Commission for their review. The agencies provided technical comments, which we incorporated as appropriate. Representatives from Apple, the Association of Public Health Laboratories, Google, Massachusetts Institute of Technology’s Lincoln Laboratory, and MITRE Corporation also reviewed a draft of this product; we incorporated their technical comments as appropriate. We are sending copies of this report to the appropriate congressional committees and other interested parties. In addition, the report is available at no charge on the GAO website at https://www.gao.gov. If you or your staff have any questions about this report, please contact Karen L. Howard at (202) 512-6888 or howardk@gao.gov or Vijay A. D’Souza at (202) 512-6240 or dsouzav@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Karen L. Howard, PhD Director, Science, Technology Assessment, and Analytics Vijay A. D’Souza Director, Information Technology and Cybersecurity Exposure Notification GAO-21-104622 46 List of Congressional Addressees The Honorable Patrick Leahy The Honorable Bennie G. Thompson Chairman Chairman The Honorable Richard Shelby The Honorable John Katko Vice Chairman Ranking Member Committee on Appropriations Committee on Homeland Security United States Senate House of Representatives The Honorable Ron Wyden The Honorable Carolyn B. Maloney Chairman Chairwoman The Honorable Mike Crapo The Honorable James Comer Ranking Member Ranking Member Committee on Finance Committee on Oversight and Reform United States Senate House of Representatives The Honorable Patty Murray The Honorable Richard Neal Chair Chairman The Honorable Richard Burr The Honorable Kevin Brady Ranking Member Republican Leader Committee on Health, Education, Labor, Committee on Ways and Means and Pensions House of Representatives United States Senate The Honorable Morgan Griffith The Honorable Gary C. Peters Republican Leader Chairman Subcommittee on Oversight and The Honorable Rob Portman Investigations Ranking Member Committee on Energy and Commerce Committee on Homeland Security and House of Representatives Governmental Affairs United States Senate The Honorable Brett Guthrie Republican Leader The Honorable Rosa L. DeLauro Subcommittee on Health Chairwoman Committee on Energy and Commerce The Honorable Kay Granger House of Representatives Ranking Member The Honorable Michael C. Burgess Committee on Appropriations House of Representatives House of Representatives The Honorable Frank Pallone, Jr. Chairman The Honorable Cathy McMorris Rodgers Republican Leader Committee on Energy and Commerce House of Representatives Exposure Notification GAO-21-104622 47 Appendix I: Objectives, Scope, and Methodology Objectives • Federal Communications Commission (FCC). We were asked to assess smartphone applications (apps)—commonly referred to as In addition, we interviewed representatives exposure notification apps—that are intended from companies involved in the development to notify persons of potential exposure to of exposure notification apps (Google, Apple, infectious diseases. This report discusses: and PathCheck Foundation); public health organizations (Association of Public Health • the benefits and design of exposure Laboratories, Public Health Informatics notification apps; Institute, Council of State and Territorial Epidemiologists, Association of State and • the current level of deployment in the Territorial Health Officials); federally funded U.S.; research and development centers • challenges affecting their use; and (Massachusetts Institute of Technology’s • policy options that could help address key Lincoln Laboratory and MITRE Corporation); challenges for future use. and academic researchers from Oxford University’s Big Data Institute, Nuffield Scope and methodology Department of Medicine. We identified these entities through our interviews and document reviews. During our interviews with officials To address these objectives, we reviewed and representatives, we discussed topics such documentation and met with officials from as exposure notification app functionality; selected federal agencies and entities benefits of its use; levels of deployment in the involved in providing guidance, funding U.S.; technological limitations; and challenges research, and other efforts related to to its development, deployment, and use. We exposure notification apps. These agencies also obtained written responses from two were: organizations: the National Association of County and City Health Officials and Linux • Centers for Disease Control and Foundation Public Health. Prevention (CDC) and National Institutes of Health (NIH) within the Department of Health and Human Services (HHS), Further, we conducted a literature search for • Cybersecurity and Infrastructure Security articles regarding exposure notification apps, Agency, Science and Technology including their benefits, capabilities, and Directorate, within the Department of challenges, as well as policy options Homeland Security (DHS), associated with the apps. A research librarian • National Institute of Standards and conducted searches of various databases Technology (NIST), Federal Trade including Inspec, Scopus, Policy File, Commission (FTC), within the Department ProQuest’s COVID-19 Research Database, and of Commerce, and the Harvard Kennedy School’s Custom Google Think Tank Search. We used synonyms of the Exposure Notification GAO-21-104622 48 following search terms to identify relevant We also contacted several individual states to articles: contact tracing, exposure verify their status in deploying an app. We notification, Google Apple exposure, analyzed the inventory to identify the extent application, app, system, platform, digital, to which states and territories had deployed mobile, smartphone. We paired these search apps, the underlying technologies used (e.g., terms with additional synonyms for privacy, Bluetooth Low Energy), and the use of security, policy, legislation, opportunities, and national servers. 87 States that had an app in a challenges. We considered articles that met pilot phase at the time of our review were the following criteria: published from 2016 included in the category of “states that had through January 2021 in academic journals, not deployed an app as of June 2021.” working papers, trade journals, legislative materials, and reports by government To obtain additional information associated agencies and nonprofit organizations. From with the development and use of the apps, the results produced by this search, we we interviewed state public health officials reviewed a selection of articles to provide an from a non-generalizable sample of nine overview and additional context for our states that had an exposure notification app research objectives. We also used the results as of January 1, 2021: Alabama, Colorado, to help inform our development of an Connecticut, Minnesota, Nevada, North inventory of states by app deployment status, Carolina, Pennsylvania, Virginia, and among other sources noted below. Washington. 88 We selected this sample based on deployment date, geographical To identify the current level of deployment in distribution, the number of COVID-19 cases the U.S., we developed an inventory of and deaths, and app developer. We aimed for exposure notification apps that had been a selection of states that would allow for the deployed by U.S. states and territories as of selected states to provide a broad overview June 2021. We developed the inventory by: and context for assessing our engagement’s research objectives. Because the selection • reviewing inventories that had been was based on a non-generalizable sample, the developed by other organizations; 86 results cannot be used to make inferences • reviewing state health department about all states that had deployed an app. We websites related to COVID-19 to identify also received written feedback to structured whether they identified an available app, questions from two additional states or plans to deploy one; (Louisiana and Utah) that deployed apps in • conducting Google searches; and the later stages of our evidence collection. • reviewing Android and iPhone app stores. We also conducted a review of each of the selected states apps on both a phone using 86The inventories included those developed by MIT Technology 88We considered a state as having an app if it had an official Review, the Association of State and Territorial Health Officials, application available for download or the state officially and the Ada Lovelace Institute. supported an exposure notification system that exists in a 87Specifically, whether states used the Association of Public smartphone operating system. As of January 1, 2021, we had identified 20 states with an exposure notification app. After we Health Laboratories’ National Key Server and Multi-tenant selected our sample, we learned that one additional state— Verification Server. Wisconsin—had deployed an app in late December 2020. Exposure Notification GAO-21-104622 49 the iOS (iPhone 6) and Android (Samsung intended to be inclusive of all potential policy Galaxy S9) operating systems. As a part of this options and are neither recommendations to review, we reviewed the general functions, federal agencies nor matters for features, and usability of the apps. To help congressional consideration. They are also not understand how privacy considerations listed in any specific rank or order. We are not applied to the apps, we examined and suggesting that they be done individually or compared each state’s privacy policies with combined in any particular fashion. recommended practices identified in federal Additionally, we did not conduct work to guidance, such as CDC’s guidelines for digital assess how well they may lead to a particular tools. 89 outcome. In addition, to obtain perspectives from states We conducted our work from November 2020 that had not deployed an app, we collected to September 2021 in accordance with all information from a non-generalizable sections of GAO’s Quality Assurance selection of seven states that had not Framework that are relevant to technology deployed an app at the time of our review assessments. The framework requires that we (Montana, Nebraska, Oregon, Rhode Island, plan and perform the engagement to obtain South Carolina, Texas, and West Virginia), sufficient and appropriate evidence to meet which included an interview with one state our stated objectives and to discuss any and written responses to a semi-structured limitations to our work. We believe that the set of questions for the other six. We selected information and data obtained, and the these states based on geographical analysis conducted, provide a reasonable distribution, suggestions from stakeholders basis for any findings and conclusions in this we interviewed, and information we gathered product. during our review regarding challenges certain states may have faced. We identified policy options based on our literature review and interviews with federal agencies, the selected states, and other stakeholders, including national health organizations and researchers. We assessed each policy option by identifying potential benefits and considerations of implementing them, as identified over the course of our review. Based on the evidence collected, we identified four policy options. The list is not 89 Centers for Disease Control and Prevention, Preliminary Criteria for the Evaluation of Digital Contact Tracing Tools for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and Guidelines for the Implementation and Use of Digital Tools to Augment Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec. 15, 2020). Exposure Notification GAO-21-104622 50 Appendix II: Exposure Notification App Adoption Rates for Selected U.S. States As previously discussed, states may use the rate of adoption to measure the success of their efforts to promote exposure notification apps. However, officials from the 11 states in our review reported that they used differing methods to calculate adoption rates. App adoption rates can be determined by dividing the total number of smartphone downloads, or activations, (numerator) by the size of a given population (denominator). However, states use different methods for determining the denominator, which affects the adoption rates. Specifically, two states used the total state population, three used populations aged 18 or older, and three used the percent of the population with a smartphone (either age 18 or older or 18 to 65), according to state officials. These inconsistent methods impede comparative assessments across states. The following table provides information on the apps deployed by the nine U.S. states selected for our review, plus Louisiana and Utah. Table 4: Reported exposure notification app adoption rates for 11 U.S. states, as of June 2021 State Type App name Launch Reported Reported Reported (custom or date downloads and/or population adoption Express) estimated (denominator) rateb activationsa (numerator) Alabama Custom GuideSafe™ Aug. 280,000 Not provided 20% 2020 Colorado Express COVID Oct. 2,511,070 Not provided 42% Exposure 2020 The combined Notifications number of iOS and Android activations taken from several sources. Connecticut Express COVID Alert Nov. N/A N/A N/A CT 2020 Louisiana Custom and COVID Jan. 663,379 3,560,976 19% Express Defense 2021 This includes the (18 and older) combined number Based on U.S. of iOS and Android Census Bureau app downloads for estimate for the the custom app, as adult population. well as the Android app downloads and the estimated number of activations for iOS for the Express option. Exposure Notification GAO-21-104622 51 Minnesota Custom and COVIDAware Nov. 1,419,232 5,600,000 25% Express 2020 This includes the (Total population) combined number Based on U.S. of iOS and Android Census Bureau app downloads for estimates. the custom app, as well as the Android app downloads and the estimated number of activations for iOS for the Express option. Nevada Custom and COVID Trace Aug. 1,202,874 3,100,000 49% Express 2020 This includes the (80% of state combined number population, a of iOS and Android proxy for the app downloads for number of people the custom app, as with well as the smartphones) Android app Based on U.S. downloads and Census Bureau the estimated estimates. number of activations for iOS for the Express option. North Custom SlowCOVIDNC Sept. 854,802 9,472,502 9% Carolina 2020 The combined (18 and older) number of iOS and Based on North Android app Carolina’s Office downloads. of State Budget and Management estimates for the adult population. Pennsylvania Custom COVID Alert Sept. 912,863 10,880,000 8% PA 2020 The combined (18-65 years old) number of iOS and Based on U.S. Android app Census Bureau downloads. estimates for the adult population and Pew Research Center estimates that 85% of the adult population has a smartphone. Utah Express UT Exposure Feb. N/A N/A N/A Notifications 2021 Exposure Notification GAO-21-104622 52 Data are not tracked. However, officials estimated there were 600,000 activations in the initial phase of app deployment. Virginia Custom and COVIDWISE Aug. 1,100,338 4,253,335 26% Express 2020 This includes the (18-65 years old) combined number Based on U.S. of iOS and Android Census Bureau app downloads for estimates for the the custom app, as adult population well as the and Google and Android app Apple statements downloads and that 80% of the the estimated adult population number of has a smartphone. activations for iOS for the Express option. Washington Express WA Notify Nov. 2,068,916 6,115,500 34% 2020 The number of (18 and older) Android app Based on downloads and Washington’s the estimated Office of Financial number of Management activations for iOS. estimates for the adult population and Pew Research Center estimates that 85% of the adult population has a smartphone. Legend: N/A = A state that does not track the number of app downloads or its adoption rate. Source: GAO compilation of data from selected states, related documents, interviews and other sources. I GAO-21-104622 Notes: The “as of “date for the data provided by the states ranged from May 31, 2021 to June 11, 2021. aFor states using the Express option, daily activations for Android are from the Google Play app store, while the Apple activations numbers are an estimate based on counting the number of times people accessed the app’s logo image and using a multiplier provided by Google and Apple to estimate the number of people who go on to install the Express option. bThe adoption rates were provided to us by the states; we rounded the rates up to the nearest whole number. We did not calculate the adoption rates; however, the rates can be calculated with the provided numerators and denominators for the states that provided this information. Exposure Notification GAO-21-104622 53 Appendix III: GAO Contacts and Staff Acknowledgments GAO contacts Karen L. Howard, PhD, (202) 512-6888 or howardk@gao.gov Vijay A. D’Souza, (202) 512-6240 or dsouzav@gao.gov Staff acknowledgments In addition to the contacts named above, Sushil Sharma (Assistant Director), Neela Lakhmani (Assistant Director), Eric Bachhuber (Analyst in Charge), Scott Borre (Analyst in Charge), Nora Adkins, Daniel Emirkhanian, Donna Epler, Nancy Glover, Anika McMillon, Melissa Melvin, Monica Perez-Nelson, Ben Shouse, Amber Sinclair, and Umesh Thakkar made key contributions to this report. Rebecca Gertler, Tim Kinoshita, Eleni Orphanides, and Ethiene Salgado-Rodriguez also contributed to this report. (104622) Exposure Notification GAO-21-104622 54 GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to https://www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts and read The Watchblog. Visit GAO on the web at https://www.gao.gov. To Report Fraud, Waste, and Abuse in Federal Programs Contact: Website: https://www.gao.gov/fraudnet/fraudnet.htm Automated answering system: (800) 424-5454 or (202) 512-7470 Congressional Relations Nikki Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Public Affairs Chuck Young, Managing Director, YoungC1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149, Washington, DC 20548 Strategic Planning and External Liaison Stephen Sanford, Managing Director, spel@gao.gov, (202) 512-9715 U.S. Government Accountability Office, 441 G Street NW, Room 7B37N, Washington, DC 20548