U.S. Department of Health and Human Services Office of Inspector General + CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain Ann Maxwell Deputy Inspector General for Evaluation and Inspections January 2024, OEI-06-22-00030 U.S. Department of Health and Human Services Office of Inspector General Report in Brief January 2024, OEI-06-22-00030 Why OIG Did This Review CDC Has Improved the Nursing Home Reporting The National Healthcare Safety Process for COVID-19 Data in NHSN, but Network (NHSN) has served as a critical source for monitoring the Challenges Remain effects of the COVID-19 pandemic, and informing the Federal, State, Key Takeaway What OIG Found and local pandemic response. In Despite CDC efforts, both CDC and CDC struggled to support May 2020, the Centers for nursing homes experienced difficulties nursing homes during mass Medicare & Medicaid Services during a mass enrollment of more than enrollment into NHSN in 2020 (CMS) issued a requirement for 12,000 facilities into NHSN to begin and as COVID-19 reporting nursing homes to report COVID-19 reporting COVID-19 data in May 2020. requirements changed data to NHSN. The Centers for throughout the pandemic. As the pandemic continued, CDC added Disease Control and Prevention CDC has improved the nursing data variables to NHSN, including fields (CDC) has operated NHSN since home reporting process and with personally identifiable information, 2005, but nursing home reporting guidance, but some in response to emerging data needs and had been voluntary, with challenges remain. Continued new Federal reporting requirements. participation from only a small improvements to NHSN user Nursing homes had to upgrade their proportion of facilities. The support and data quality will security access levels to report the reporting requirement resulted in be important for continued sensitive data. At this time, CDC the influx of thousands of nursing reporting on vaccinations and experienced a significant backlog of homes enrolling in and reporting for future public health support requests, which also inhibited to NHSN in 2020, while they, and surveillance. some facilities from accessing NHSN. CDC, also responded to the pandemic. CDC improved the process of nursing home reporting to NHSN throughout the pandemic. Facilities acknowledged this effort and reported that CDC This evaluation provides insights support improved, but some continued to experience difficulty getting into nursing home experiences assistance. Additionally, a quarter of nursing homes reported lacking enrolling in and reporting to confidence in the quality of NHSN data, despite the quality assurance checks NHSN, and CDC efforts to facilitate CDC conducts on key variables. reporting such as user support for facilities facing difficulties. These After December 2024, CMS reporting requirements for some key variables insights can help CDC address will expire, but the mandate for reporting vaccination-related data will ongoing challenges, and mitigate remain. CDC stated that it will continue to support voluntary reporting of potential issues in future updates COVID-19 data and other infection and quality measures, and modernize or expansions. NHSN reporting processes. Stakeholders and CDC expressed that having nursing home participation in NHSN is valuable for public health surveillance, How OIG Did This Review and the agency is exploring opportunities to leverage the current national We administered an electronic enrollment for reporting on other health outcomes. survey to a simple random sample of 197 nursing homes from a What OIG Recommends and How the Agency Responded population of 15,324 facilities that To continue improvements, we recommend that CDC (1) improve the user have reported COVID-19 data to support the NHSN Help Desk provides to nursing homes, (2) take further NHSN, and interviewed a subset of steps to ensure the quality of nursing home reporting of COVID-19 data to facilities. We also interviewed CDC NHSN, and (3) consider how quality assurance checks can be enhanced to and CMS officials to understand ensure data accuracy, as appropriate. CDC partially concurred with our first CDC efforts to facilitate nursing recommendation and concurred with our second and third home enrollment and reporting to recommendations. NHSN. We based our findings on analysis of survey and interview responses. TABLE OF CONTENTS BACKGROUND................................................................................................................................................ 4 FINDINGS ....................................................................................................................................................... 14 Despite CDC efforts, both the agency and nursing homes experienced difficulties during mass enrollment in NHSN: May–June 2020 ............................................................................................................................. 14 CDC changes to NHSN data fields and security resulted in new challenges for the agency and nursing homes: August–December 2020 ....................................................................................................................................... 18 CDC has improved nursing home experiences reporting to the COVID-19 Module throughout the pandemic.................................................................................................................................................................................... 21 Nursing home experiences with reporting have improved since May 2020, though issues remain for both facilities and CDC ......................................................................................................................................................... 23 CDC has made efforts to ensure data quality, but nursing homes reporting to the COVID-19 Module expressed concerns ................................................................................................................................................................ 28 CDC plans to continue improving NHSN for nursing home reporting and is considering how to leverage national enrollment for voluntary reporting on other health events ............................................... 30 CONCLUSION AND RECOMMENDATIONS .......................................................................................... 33 Improve the user support the NHSN Help Desk provides to nursing homes ................................................. 33 Take further steps to ensure the quality of nursing home reporting of COVID-19 data to NHSN ......... 34 Consider how quality assurance checks can be enhanced to ensure data accuracy, as appropriate ..... 34 AGENCY COMMENTS AND OIG RESPONSE ........................................................................................ 35 APPENDICES................................................................................................................................................. 36 Appendix A: Estimates, Confidence Intervals, and Key Statistics for Nursing Home Survey Results ...... 36 Appendix B: Changes to the LTCF COVID-19 Module .............................................................................................. 43 Appendix C: Step-by-Step Enrollment Process for NHSN Users .......................................................................... 44 Appendix D: SAMS Registration and Identity Verification ...................................................................................... 47 Appendix E: Agency Comments ........................................................................................................................................ 48 ACKNOWLEDGMENTS AND CONTACT ................................................................................................ 53 ABOUT THE OFFICE OF INSPECTOR GENERAL.................................................................................... 54 ENDNOTES ................................................................................................................................................... 55 BACKGROUND OBJECTIVES 1. To examine nursing home experiences enrolling in and reporting required data to the National Healthcare Safety Network (NHSN) COVID-19 Module. 2. To assess Centers for Disease Control and Prevention (CDC) efforts to facilitate nursing home reporting to NHSN. The COVID-19 pandemic has had a devastating impact on nursing homes and their residents. Nursing homes have experienced widespread challenges during the pandemic, such as inadequate access to personal protective equipment (PPE) and staffing shortages. 1, 2 As of January 1, 2023, more than 1.4 million nursing home residents in the United States had confirmed cases of COVID-19, with approximately 160,000 deaths. 3 To monitor the effects of the pandemic and inform public health policies, the Centers for Medicare & Medicaid Services (CMS) issued a requirement for nursing homes to report COVID-19 data to CDC's NHSN. 4, a The reporting requirement resulted in an influx of thousands of nursing homes submitting data to NHSN. While the data have been critical for informing the pandemic response, nursing homes have experienced challenges enrolling in and reporting COVID-19 data to NHSN. The National Healthcare Safety Network CDC launched NHSN, a web-based surveillance system, in 2005 to collect data on health care-associated infections (HAIs) in health care facilities. 5 The Surveillance Branch, within CDC's Division of Healthcare Quality Promotion, manages NHSN. 6 NHSN's designated operating budget was $21 million in fiscal year 2022. 7 NHSN Long-term Care Facility Component CDC launched the NHSN Long-term Care Facility (LTCF) Component in 2012 for voluntary reporting of HAI and prevention data. 8 The LTCF Component allows facilities to monitor HAIs and infection control to identify problems, improve care, and determine progress towards national HAI goals. 9 Several types of LTCFs report to the component, including nursing homes and assisted living facilities. 10, b The LTCF _____________________________________________________________________________________________________________________________ aIn this report, the term "nursing homes" refers to both Medicare-certified skilled nursing facilities and Medicaid-certified nursing facilities. See SSA § 1819, "Requirements for, and assuring quality of care in, skilled nursing facilities," and SSA § 1919, "Requirements for nursing facilities." bNursing homes, assisted living, and Intermediate/Chronic Care Facilities for individuals with intellectual disability report to the NHSN LTCF Component. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 4 Component consists of four modules: (1) the HAI Module, (2) the Laboratory-identified Event Module, (3) the Prevention Process Measures Module, and (4) the COVID-19 Module. 11, 12 CDC released the LTCF COVID-19 Module in April 2020 to support pandemic response activities. 13, 14 Initially, the COVID-19 Module only collected data on the impact of the pandemic on residents and staff, facility supplies, and the facility response. 15, 16 CDC has since made changes to the module to collect data on various pandemic-related developments. 17, 18, 19, 20 (See Exhibit 1 for the structure of the COVID-19 Module, as of June 2023, and Appendix B for a timeline of major changes to the module.) Exhibit 1: Structure of the LTCF COVID-19 Module Source: CDC, LTCF COVID-19 Module. Accessed at https://www.cdc.gov/nhsn/ltc/covid19/index.html on February 6, 2023. CDC, HCP & Resident COVID-19 Vaccination. Accessed at https://www.cdc.gov/nhsn/ltc/weekly-covid-vac/index.html on December 3, 2021. Note: The bottom three boxes under the Surveillance Reporting Pathways (Supplies and PPE, Ventilator Capacity and Supplies, and Therapeutics) indicate pathways CDC later removed from the COVID-19 Module. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 5 COVID-19 Module Enrollment LTCFs must enroll in NHSN to use the COVID-19 Module. 21 Facilities that were already enrolled in the LTCF Component had immediate access to the module. Facilities newly enrolling in NHSN have followed a multi-step enrollment process to gain access. Each LTCF must designate an NHSN Facility Administrator (NHSN Administrator) for enrollment. 22 The NHSN Administrator serves as the facility's point of contact for receiving information from and reporting data to NHSN. 23 The NHSN Administrator submits all required information in the NHSN registration page. 24 Following registration with NHSN, the applicant receives an email invitation to register with the Secure Access Management Service (SAMS) with steps to submit facility contact information. 25 The NHSN Administrator then enters all required information in SAMS to complete enrollment. 26 (See Appendix C for the step-by-step enrollment process published by CDC.) SAMS Registration SAMS is an online portal that provides authentication, login, and secure access to authorized personnel to more than 300 CDC technology applications, including NHSN. 27 The Digital Services Office (Digital Services), within the CDC Office of Chief Information Officer, manages SAMS. 28 NHSN applicants undergo an identity verification process in SAMS, due to the sensitivity of the data they will be submitting. SAMS grants users one of three distinct access levels following registration. Each tier requires increasing levels of user authentication during enrollment and login. • Level 1: Users are not authorized to report personally identifiable information. 29 This access level does not require identity verification or any additional security measure during the enrollment or login process. 30 • Level 2: Applicants must complete identity verification during enrollment. 31 Users may log in to the application with only the username and password. 32 • Level 3: Users are authorized to report personally identifiable information. 33 Users must complete identity verification during enrollment. 34 To log in, users must complete second-factor authorization using a soft token, which is a mobile authentication application, or a hard token, which is a physical card mailed to the user's home address. 35, 36 According to CDC, reporting to NHSN requires SAMS Level 3 access if the user reports data that includes personally identifiable information. c _____________________________________________________________________________________________________________________________ cInitially, nursing homes newly enrolling in the COVID-19 Module only needed SAMS Level 1 access, as reporting to the module did not involve submitting data with personally identifiable information. (See "Simplified enrollment process" on page 15.) CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 6 NHSN Technical Support CDC provides support to NHSN users through the NHSN User Support Help Desk (NHSN Help Desk). 37 Users submit support requests exclusively via email, given that the NHSN Help Desk does not have a user support telephone line. The staff operating the Help Desk triage tickets manually. After submitting an email request, users receive an automated response, which includes a ticket number and the expected wait time. Prior to the pandemic, user support requests to the Help Desk had an expected wait time of 2 to 5 business days, with a 7-day wait time if the inquiry required subject-matter expertise. 38 CMS Nursing Home Reporting Requirements CMS sets and enforces health and safety standards, and oversees nursing home compliance with Medicare and Medicaid standards to ensure resident health and safety. 39 On May 8, 2020, as a part of its pandemic response, CMS issued an interim final rule that required nursing homes to report COVID-19 data to NHSN. 40 Nursing homes had until May 17, 2020, to enroll in NHSN to begin reporting COVID-19 data. Nursing homes fulfill the reporting requirements via the LTCF COVID-19 Module's Surveillance Reporting Pathways (referenced in Exhibit 1 on page 5). The interim final rule required facilities to electronically report the following data on a weekly basis: • Suspected and confirmed COVID-19 infections among residents and staff, including residents previously treated for COVID-19; • Total deaths and COVID-19 deaths among residents and staff; • PPE and hand hygiene supplies in the facility; • Ventilator capacity and supplies in the facility; • Resident beds and census; • Access to COVID-19 testing while the resident is in the facility; and • Staffing shortages. 41 CDC experienced a substantial rise in NHSN enrollment by nursing homes as a result of the CMS reporting requirements. Prior to the pandemic, nursing home reporting to NHSN was largely voluntary and not Federally required. Between March and June 2020, the total number of LTCFs enrolled in NHSN increased by more than 450 percent, from 3,086 to 17,248 facilities. 42, 43, d _____________________________________________________________________________________________________________________________ d The number of LTCFs enrolled in NHSN increased by 14,162 between March and June 2020. According to CDC, the majority of the increase in NHSN enrollment is attributable to the addition of more than 12,000 nursing homes. During the week of June 7, 2020, a total of 15,365 nursing homes reported COVID-19 data to NHSN. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 7 CMS expanded its reporting requirements in a second interim final rule issued on May 13, 2021. Nursing homes fulfilled the additional requirements via the COVID-19 Vaccinations Module and the Therapeutics Pathway (referenced in Exhibit 1 on page 5). CMS added the following data variables, some of which include personally identifiable information: • COVID-19 vaccine status of residents and staff, including total numbers of residents and staff; numbers of vaccinated residents and staff; numbers of each dose of the COVID-19 vaccine received; and COVID-19 vaccination-related adverse events; and • Therapeutics administered to residents for treatment of COVID-19. 44 Enforcement Actions CMS has the authority to penalize nursing homes that do not meet the weekly reporting requirements. On September 2, 2020, CMS issued an interim final rule that established civil money penalties for facilities that fail to submit required data for a given reporting week. 45 CMS finds that a nursing home meets requirements if the facility reports any data related to the effects of COVID-19 on residents and staff. 46 Nursing Home Reporting The number of data points nursing homes are required to submit to NHSN has changed over time. In November 2021, facilities were required to submit up to 132 data points, including 100 data points that were conditionally required on the basis of facility responses. 47 For example, if a nursing home reported that it had any residents stay at the facility at least 1 day (24 hours) of a reporting week, the nursing home needed to submit up to six additional data points about resident vaccination status. Over time, CDC has substantially reduced the number of questions a nursing home may need to submit. As of August 2023, nursing homes only have to submit up to 37 data points, 3 of which are conditionally required. Of the 34 required data points, 5 are related to infections and deaths, and 29 are related to vaccinations. 48, 49, 50, 51 CMS requirements to report data related to COVID-19 infections, deaths, and staffing will expire after December 2024, but requirements for nursing homes to report vaccination-related data will continue. 52 According to CDC, this means that, beginning in 2025, facilities will only report data for the 29 vaccination-related variables. Nursing homes report data to the COVID-19 Module by entering data directly into the module or uploading data files based on templates provided by CDC.53 The module visually shows when a submission is complete or is missing data points. 54 Authorized group users, such as State and local health departments, may also report to NHSN on behalf of nursing homes, but facilities are still responsible for data submission. 55 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 8 Quality Assurance Checks CDC conducts quality assurance (QA) checks on the data for facility admissions, COVID-19 cases, and COVID-19 deaths in the COVID-19 Module. 56 If a nursing home's data fail to meet the QA criteria, CDC flags the data to allow for correction by the facility. For resident and staff variables, the data must: • Represent a weekly count of a given variable and not a cumulative total, and • Not exceed a subset of cutoff values established by CDC to identify outliers that appear invalid. Following its QA checks, CDC shares nursing home COVID-19 data from NHSN with CMS on a weekly basis. Prior to sharing the file, CDC nullifies data that failed its QA checks, and flags the facility in the dataset for CMS. CDC also conducts QA checks on two vaccination variables: the percentage of current residents, and the percentage of health care personnel, with no medical contraindications who received a completed or partial COVID-19 vaccination at any time. For the given reporting week, the data values for the two vaccination variables cannot be less than or equal to 10 percent. 57 Use of NHSN COVID-19 Data Nursing home COVID-19 data from NHSN have informed efforts within the U.S. Department of Health and Human Services (HHS) and the broader stakeholder community. According to CDC, other stakeholders that used COVID-19 data from NHSN include the following: • The White House COVID-19 Task Force, to inform its emergency response actions; • The Assistant Secretary of Preparedness and Response, with efforts to distribute PPE and other medical supplies, as well as to address health care personnel shortages; • The Federal Emergency Management Agency, to also address health care personnel shortages; • Long-term care trade or membership organizations, to support outreach and educational efforts; • State and local health departments, to inform prevention activities and pandemic outbreak investigations; and • CMS, to support efforts by its Quality Improvement Organizations, among other uses. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 9 CMS referenced NHSN data on staff COVID-19 vaccination coverage in its November 2021 interim final rule regarding vaccination requirements for health care personnel. 58 CDC also posts data specific to COVID-19 cases and deaths among nursing home residents and staff on a publicly available dashboard that is updated weekly. 59 Since May 31, 2020, CMS has maintained a public dataset of weekly nursing home COVID-19 data to support efforts to protect the health and safety of residents and other consumers. 60, 61 CMS conducts separate QA checks on the NHSN data it receives from CDC to review compliance with certain requirements, such as reporting data on total resident and staff COVID-19 deaths. 62 If a nursing home's data fail to meet CMS's QA criteria, CMS does not include any of the facility's data in the publicly posted dataset, and notes that the facility did not pass the QA check. CMS has since added to the ways it presents NHSN data to consumers and other stakeholders. In September 2021, CMS began displaying COVID-19 vaccination data on its Care Compare tool, which provides quality metrics for nursing homes and other health care facilities to help consumers make more informed choices.63 Related Work In September 2021, OIG released a report that found that COVID-19 data submitted to NHSN by a small portion of nursing homes in the early months of the CMS interim final rule were inaccurate or incomplete. Specifically, OIG found that about 5 percent of reported nursing home data between June 22 and July 19, 2020, did not include all required data and were not complete or accurate following CMS QA checks. 64 OIG has other work related to the pandemic's effects on nursing homes and Federal and State programs. OIG found that overall mortality in nursing homes increased in 2020, and that States conducted fewer onsite surveys to assess compliance with Federal requirements during the pandemic. 65, 66 OIG also found that 1,358 nursing homes had extremely high infection rates-75 percent or more-during surges of COVID-19 cases in the spring and fall of 2020, and experienced an average overall mortality rate approaching 20 percent. 67 An ongoing study is examining nursing home strategies to address challenges during the pandemic and in future public emergencies. 68 Methodology Scope of Inspection We examined the experience of nursing homes enrolling in and reporting required data to the LTCF COVID-19 Module, including any challenges facilities faced, and strategies used to overcome the difficulties. We did not assess the completeness or accuracy of the data. We also evaluated CDC efforts to facilitate nursing home enrollment in and reporting to the COVID-19 Module. These efforts included CDC instructions and correspondence about NHSN, and its actions to address nursing CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 10 home challenges. Although we describe CMS involvement with the module, we did not evaluate CMS actions regarding NHSN as part of this evaluation. The timeframe for our review was February 2020 through June 2023. We used the following data sources for our evaluation: (1) an electronic survey of a sample of LTCFs reporting to the COVID-19 Module; (2) interviews with a smaller subset of nursing home administrators and staff, selected from the survey sample; (3) interviews with CDC officials involved with NHSN; (4) interviews with CMS officials involved in collaborating with CDC regarding nursing home COVID-19 data in NHSN; and (5) documentation provided by CDC regarding nursing home enrollment and reporting to the COVID-19 Module. Data Collection Survey. We administered an electronic survey to a simple random sample of 200 LTCFs and received responses from 171 facilities, for an 86-percent rate of contact. e, f Our sample population consisted of 15,324 facilities that reported data at least once to the LTCF COVID-19 Module between May 24 and December 26, 2021. (Our sample of 200 LTCFs represented 1.3 percent of the total population.) Of the 171 respondents, 167 were nursing homes and the remaining 4 were LTCFs of other types. g Our sampling method and response rate allowed us to project the results of our survey to the total population of LTCFs reporting to the COVID-19 Module during this timeframe. The survey consisted of both closed- and open-ended questions. The closed-ended questions focused on nursing home experiences enrolling in and reporting data to the COVID-19 Module. The open-ended questions asked nursing homes to provide additional details on any identified challenges and improvements to enrollment and reporting, as well as perceptions of related CDC efforts. Interviews. We conducted semi-structured virtual interviews with stakeholders involved in nursing homes enrolling in and reporting to the COVID-19 Module as follows: • Individual interviews with 12 CDC officials-including Division of Healthcare Quality Promotion leadership involved with NHSN, the COVID-19 Module, and the NHSN Help Desk, and Digital Services leadership involved with SAMS-to identify and understand CDC efforts to facilitate nursing home enrollment in and _____________________________________________________________________________________________________________________________ eWe randomly selected a sample of 200 facilities from the population of 15,324 that reported to the LTCF COVID-19 Module at least once between May 16 and December 26, 2021. We did not distribute a survey to three nursing homes, as two were involved in ongoing work by OIG's Office of Investigations, and the remaining one had permanently closed. fWe made a minimum of three follow-up attempts to contact respondents who did not submit a completed survey. g We refer to the survey respondents as nursing homes for the remainder of our report. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 11 reporting to NHSN. We also inquired about related challenges CDC experienced, as well as factors that may negatively affect reporting. • Group interview with eight CDC officials as a follow-up discussion to clarify CDC's most recent NHSN efforts and future uses of NHSN in collaboration with stakeholders. We also discussed ongoing reporting requirements and how they would affect nursing home engagement with NHSN. • Individual interviews with representatives from 10 facilities involved with enrolling in and reporting to the COVID-19 Module, representing a subset of survey respondents, to gain further insight on nursing home experiences with NHSN. Using facility survey responses, we selected facilities that (1) reported, and provided detail about, common themes that we identified in our analysis of the overall survey results; and (2) reported challenges and navigated those challenges in ways we found informative. We conducted interviews with 12 respondents representing the 10 facilities. • Group interview with 12 CMS officials involved with nursing home reporting to the COVID-19 Module, to identify inter-agency collaboration efforts. Documentation. We reviewed documents related to nursing home reporting to the COVID-19 Module, including (1) CDC manuals, memos, presentations, and other guidance about enrolling in and reporting to the module; (2) records of user support requests submitted to and resolved by the NHSN Help Desk, and templates for related email communication; and (3) CDC guidance that explains or relates to QA checks performed on the data. Data Analysis Survey. We analyzed survey results to gain insight into nursing home experiences reporting to the COVID-19 Module and perceptions of CDC user support. We used closed-ended question responses to identify the proportion of nursing homes with certain experiences enrolling in and reporting COVID-19 data to NHSN. Using SAS statistical software, we projected the closed-ended question responses from our nursing home respondents to the population of facilities enrolled in the COVID-19 Module at the time of data collection. The open-ended question responses provided additional context and anecdotal examples that illustrated our findings. We also used the open-ended responses to select a sample of nursing homes to interview. Interviews. Interviews with CDC and CMS officials, as well as with nursing home representatives, added important context and examples of NHSN-related challenges and CDC efforts to address them. We used interviews with CDC to detail support efforts, including how the agency formulated and distributed guidance, and how it responded to difficulties experienced by nursing homes. We used the interview with CMS to identify how CDC addressed updates to the COVID-19 Module, as well as how CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 12 it accommodated CMS use of NHSN data. We used interviews with facility representatives to gain further insight into the frequency and severity of challenges the facilities have experienced with NHSN. The scope of our evaluation focused on CDC efforts and did not include the assessment of CMS actions related to NHSN. Documentation. We reviewed documents to identify gaps in CDC guidance that may have contributed to or helped resolve nursing home difficulties, and related correspondence. Our analysis of documents evaluated how CDC approached the challenges and relayed information related to the reporting process. Limitations We did not independently verify information self-reported by CDC and nursing homes, including data submitted by nursing homes to NHSN. We also did not determine a causal relationship between CDC actions and improved nursing home reporting. Standards We conducted this study in accordance with the Quality Standards for Inspection and Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Background | 13 FINDINGS Despite CDC efforts, both the agency and nursing homes experienced difficulties during mass enrollment in NHSN: May– June 2020 CDC described the mass enrollment of nursing homes following the CMS reporting requirements as difficult and labor-intensive. It described experiencing pressure and challenges preparing for and managing the influx of newly enrolling facilities. Starting in May 2020, immediately following the CMS interim final rule, CDC onboarded thousands of nursing homes in NHSN under a compressed timeline. Nursing homes enrolling in the COVID-19 Module also faced challenges. Among newly enrolled facilities in our sample, 101 of 117 reported experiencing difficulties during the enrollment process, such as issues with accessing CDC guidance and resources.h CDC made efforts to mitigate enrollment-related challenges, including streamlining the process and reallocating staff CDC prepared for the mass onboarding of nursing homes internally, and by collaborating with external partners such as CMS. CDC frequently communicated with partners; added and relocated staff; and distributed guidance and resources to NHSN users. One CDC official explained that joint efforts with external partners were particularly important in successfully enrolling facilities in the COVID-19 Module. CDC decided to use NHSN for collecting nursing home COVID-19 data in collaboration with CMS. CDC and CMS reported that, starting in late February 2020, the two Federal agencies routinely met to discuss COVID-19-related items, particularly in regard to nursing homes. They identified the need to obtain data from nursing homes during the pandemic, and decided to use NHSN as the data collection system. CDC regularly sought CMS feedback on the language and important topics of interest for data collection. CMS concurrently developed the interim final rule, and involved CDC in writing the reporting requirements, rules, guidance, and memos. This collaboration was critical, because the development of the COVID-19 Module, and related guidance, depended on the CMS interim final rule to dictate what data nursing homes would be required to report. Both agencies reported that they were able to accomplish the module development due to collaborative efforts with external partners. On April 19, 2020, CMS alerted _____________________________________________________________________________________________________________________________ h Of the sample nursing homes, one-third (54 of 171) voluntarily enrolled in NHSN prior to the CMS interim final rule, and two-thirds (117 of 171) newly enrolled after CMS issued the reporting requirements. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 14 nursing homes to the upcoming reporting requirements. Following the issuance of the CMS interim final rule on May 8, nursing homes had until May 17, 2020, to enroll in NHSN to report COVID-19 data. CDC and CMS held national stakeholder calls with trade organizations, which contacted their members about enrollment and directed nursing homes to guidance and resources such as the NHSN Help Desk. One CMS official stated that a combination of outreach and CMS authority to require facility participation helped enrollment. A CDC official agreed, explaining that voluntary reporting had not been as effective as CMS requiring nursing homes to report to NHSN. CDC rapidly developed and launched the LTCF COVID-19 Module in April 2020. CDC built the reporting platform, and its subject-matter experts developed questions to inform pandemic response and prevention. The agency stated that discussions about creating a module in NHSN for COVID-19 reporting began in March 2020. One CDC official reported that developing and launching the COVID-19 Module itself occurred over 10 days, from April 16 to April 25. Agency officials noted that the development of new NHSN modules prior to February 2020 "normally took multiple years." Another official explained that existing system infrastructure and field expertise, in addition to the allocation of additional resources, allowed for an expedited module development process. CDC anticipated and made preparations to address the enrollment surge, including simplifying the onboarding process, allocating additional staff, and hosting support calls. Following the development of the COVID-19 Module, CDC identified potential challenges associated with enrolling a large volume of new users who are unfamiliar with NHSN and electronic reporting. According to Surveillance Branch officials, CDC used several strategies to alleviate strain during the enrollment surge, such as the following: • Simplified enrollment process. Nursing homes initially submitted data to the COVID-19 Module that consisted of aggregate, facility-level information, but not personally identifiable information. CDC created an enrollment process with SAMS Level 1 access for nursing homes newly enrolling in NHSN to report COVID-19 data. The simpler process reduced the length of enrollment from several weeks to a maximum of 2 days. • Click squad. CDC established a task force called the "click squad" to manually review and approve invitations, and to respond to facilities. The click squad consisted of Surveillance Branch staff and other agency staff not normally involved with NHSN, including detailees and contractors. • Conference calls. The Surveillance Branch also hosted conference calls to ease the burden on the NHSN Help Desk. CDC staff held the calls two to three times per week during the mass enrollment period to provide direct support to nursing homes. CDC officials reported that the conference calls were well-attended, sometimes with "hundreds of facilities" on the call. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 15 The NHSN Help Desk and other systems experienced difficulties providing support to newly enrolling facilities Mass enrollment of nursing homes caused significant strain on the NHSN Help Desk. The rapid increase in total NHSN users included thousands of facilities that had not previously interacted with NHSN and now needed additional help. The Help Desk had no phone support, so "[User volume] user support specialists often had to walk new users almost doubled through their questions via email, in addition to facilitating overnight, while we all other inquiries and enrollment activities, which resulted didn't double our in a bottleneck. According to CDC, Surveillance Branch staff overnight to staff worked 60 to 80 hours a week to manage the increased onboarding and Help Desk tickets during the deal with it." initial 6 to 8 weeks of nursing homes enrolling in the – CDC official COVID-19 Module. CDC assigned additional staff to the NHSN Help Desk in anticipation of enrollment- related challenges, but the Help Desk continued to experience difficulties. CDC reported that the agency increased work hours of existing user support specialists, reallocated other CDC contractors, detailed staff from various task forces, and pulled members from other teams to help respond to users. Still, according to one official, the NHSN Help Desk had delayed responses and a growing backlog of unresolved tickets. CDC explained that user support requests related to COVID-19 data submission had an expected wait time of 1 to 3 weeks, depending on the subject matter, as compared to a typical 2 to 5 days prior to the pandemic. The enrollment surge also impacted other CDC user support systems. The SAMS Help Desk, operated by Digital Services to provide SAMS-related support to users via email and telephone, faced difficulties as a result of NHSN users needing assistance. CDC reported that Digital Services frequently coordinated with the Surveillance Branch to address problems and to proactively mitigate challenges during influxes in enrollment. Collaboration between both teams also included discussions about NHSN-related inquiries that came to the SAMS Help Desk. The SAMS Help Desk often received NHSN-related telephone calls through its phone line. One nursing home representative recalled contacting the SAMS Help Desk with NHSN-related inquiries because it had a phone line, but being told that the facility could only receive the help it needed from the NHSN Help Desk. One Digital Services official reported that the SAMS Help Desk "felt the brunt of escalations related to urgency" in interactions with frustrated users. The official further stated that users wanting to speak with someone, but not being able to, was a "pinch point." CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 16 Nursing homes experienced difficulties enrolling in NHSN, such as with registering staff to report data Most newly enrolling nursing homes (86 percent) had challenges with various aspects of NHSN enrollment. i, j These facilities reported a range of difficulties, including challenges registering staff to input data (67 percent) and receiving technical assistance from CDC (56 percent). (See Exhibit 2.) Exhibit 2: Newly enrolled nursing homes reported difficulties enrolling in NHSN during the COVID-19 pandemic (n=117). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. Nursing homes made efforts to address the enrollment difficulties, most often proactively seeking assistance from CDC and others. Nearly half of the newly enrolling facilities (49 percent) reached out to the CDC NHSN Help Desk for assistance, and 30 percent reached out to CMS. Nursing homes also sought assistance from other sources. For example, 38 percent reached out to their corporate or facility chain, and 20 percent contacted nongovernment organizations, such as trade and membership groups, to seek assistance navigating the process. _____________________________________________________________________________________________________________________________ iSurvey respondents indicated whether they enrolled in NHSN prior to the CMS reporting requirements in May 2020. Upon indicating that they newly enrolled in the system, respondents were prompted by the survey to answer several questions about enrollment. This created a sample subset of 117 facilities. jOIG analysis of survey responses concluded that 101 of 117 newly enrolling facilities reported having difficulty with at least one of the listed enrollment aspects. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 17 CDC changes to NHSN data fields and security resulted in new challenges for the agency and nursing homes: August–December 2020 Following nursing homes enrolling in and beginning to report data to NHSN, CDC made several changes to the system to incorporate more information that HHS and stakeholders requested to inform pandemic response efforts. The newly requested data, such as point-of-care (POC) testing information, included personally identifiable information, which resulted in many nursing homes needing to upgrade their SAMS security access from Level 1 to Level 3. The transition to a higher level of security access required a more difficult and complex process (referenced in SAMS Registration on page 6). This resulted in an additional backlog of Help Desk requests. CDC worked with stakeholders to add new data fields to NHSN following pandemic-related developments CDC reported that changes in the COVID-19 response, environment, and knowledge resulted in new data needs, and that the agency expanded data elements as countermeasures became available. The development of new pandemic response efforts resulted in the immediate need for data and rapid additions to the COVID-19 Module. For example, CDC added therapeutics-related variables to determine which nursing homes obtained and used, and how successful they were in obtaining, antiviral and antibody therapies. CDC also included fields about the types of COVID-19 vaccines being administered to identify any distinctions. CDC subject-matter experts, such as doctors, nurses, and infection preventionists, developed NHSN questions as the pandemic continued. In developing the vaccine-related variables, they determined the level of detail and flexibility to build into the system for future accommodations, such as new manufacturers. CDC also collaborated with external stakeholders throughout the pandemic to update NHSN modules and data fields. CDC collaborated with CMS and other partners to add data fields to NHSN, and considered the burden to nursing homes. One CDC official reported that collaboration with stakeholders-including trade organizations and Federal, State, and local agencies-that wanted certain data was critical in adjusting to different phases of the pandemic. For example, another CDC official explained that the agency added questions related to POC tests and testing supplies after receiving requests from Federal stakeholders involved in the pandemic response. CDC also presented proposed changes to external partners to solicit feedback when unsure about potential alterations to NHSN. CMS provided input as CDC was developing additions to the COVID-19 Module. CMS reported that it provided feedback before and after CDC deployed certain changes, and about any changes CDC suggested. One CMS official stated that leadership and CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 18 technical teams from both agencies regularly met and discussed strategic needs during the development of the Vaccinations Module. After launching the new module for voluntary data submission, CDC and CMS discussed expanding the reporting requirements to mandate vaccine-related reporting. One CDC official characterized the collaboration as involving constant communication about user feedback, submitted data, and additional information that may be necessary. CDC reported taking the nursing home burden into account when implementing changes to the COVID-19 Module. The agency said that, although NHSN data collection is valuable to CDC and other stakeholders, adding reporting obligations required caution. One official noted that the "same people" submitting data are the ones "taking care of the patients." A CMS official reported that, while discussing changes to the COVID-19 Module to meet data collection needs, both agencies recognized the need to consider the burden on facilities. Another CDC official explained that making sudden changes to questions in NHSN does not promote standardized surveillance practices, because it is a significant undertaking for thousands of nursing homes to quickly alter their reporting. Changes to the COVID-19 Module resulted in additional difficulties, such as an increased backlog of requests for assistance The growing interest in patient-level COVID-19 data led CDC to prepare nursing homes to upgrade their security access levels to submit personally identifiable information to NHSN. CDC began conducting outreach to facilities about transitioning their security access level during August and September 2020. CDC explained that it used a phased approach for facility outreach, with a goal of contacting 2,500 nursing homes per week. One official said that it was difficult to persuade facilities to voluntarily convert their SAMS access level status immediately. CDC continued its outreach efforts, but nursing homes did not upgrade their security access levels evenly over time. On October 19, 2020, HHS issued a requirement for facilities to report POC COVID-19 test results through existing channels, including NHSN.69, 70, k CDC said that the majority of nursing homes using NHSN to fulfill this requirement started the process to upgrade their security access levels near the November 2020 deadline. The reporting requirement resulted in an influx of nursing homes upgrading their security access level, with problems exacerbated by the combination of a compressed _____________________________________________________________________________________________________________________________ kNursing homes were able to fulfill this HHS requirement by reporting data (1) directly to their State, Territorial, local, and Tribal public health departments; (2) to a State or regional information exchange; (3) to NHSN; or (4) to a centralized platform other than NHSN. Reporting POC COVID-19 test results via NHSN was the CDC- and CMS-preferred method. See CDC, COVID-19 Pandemic Response, Laboratory Data Reporting: CARES Act Section 18115, March 8, 2022, p. 5. Accessed at https://www.cdc.gov/coronavirus/2019-ncov/downloads/lab/HHS-Laboratory-Reporting-Guidance- 508.pdf on February 2, 2023. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 19 timeline and the weeks-long process involved in the upgrade. According to one CDC official, the transition to higher-level security access presented more difficulties than the initial mass enrollment in NHSN. The volume of upgrades strained the SAMS infrastructure due to a bottleneck with the identity verification process. (See Appendix C for the step-by-step enrollment process published by CDC.) Many nursing homes had to restart their upgrade processes, because SAMS had workflow timers that automatically removed applicants for lack of activity after not completing identity verifications over a set period of time. According to one Digital Services official, SAMS experienced the most significant backlog of facilities during the upgrades at the end of 2020. "They were sitting on a queue of 10,000 [upgrade requests] and can only get through 100 or so per day. That's where the big issue happened." – Digital Services official The majority of newly enrolled nursing homes reported that upgrading their SAMS access levels was a difficult process Among the sample nursing homes, many reported experiencing challenges upgrading their security access level. Of the 117 newly enrolled facilities, 72 percent found upgrading their access level to be difficult-either very difficult (39 percent) or somewhat difficult (33 percent). (See Exhibit 3.) Exhibit 3: Newly enrolled nursing homes reported difficulties upgrading from SAMS Level 1 to Level 3 access (n=117). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. Note: The total sum of the percentages above does not equal 100 percent because of rounding. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 20 Multiple nursing homes reported that it took a significant amount of time for their NHSN Administrators to upgrade their security access levels, with one facility reporting that it did not receive access until a year later. A representative from another nursing home described how not receiving timely assistance from the Help Desk inhibited the facility from fulfilling that week's reporting requirements. CDC has improved nursing home experiences reporting to the COVID-19 Module throughout the pandemic Following the initial implementation and early additions to the COVID-19 Module, CDC has continued to communicate with nursing homes; make changes to reduce the reporting burden; and collaborate with CMS and other stakeholders. CDC reported that improvements to the system have been necessary to alleviate challenges for both the agency and NHSN users, such as nursing homes. CDC addressed enrollment difficulties by streamlining the process using quicker and more accessible identity verification tools. To reduce any unnecessary reporting burden, CDC also worked with CMS to determine what data are no longer critical to collect. CDC has continued to communicate with and provide guidance and resources to nursing homes Following the launch of the COVID-19 Module in April 2020, CDC has continued to produce and regularly update online webpages, interactive webinars, virtual trainings, and various guidance documents related to nursing home reporting. CDC recorded and uploaded training resources online, and collaborated with external partners to provide and disseminate additional resources, such as responses to frequently asked questions. CDC reported using multiple channels of outreach to share information about the COVID-19 Module. The agency has maintained a list of active email addresses, collected during the enrollment process, for every NHSN Administrator submitting data to the system. Throughout the pandemic, the agency sent emails to notify facilities of new guidance and resources, such as details for upcoming trainings, and changes or updates to the COVID-19 Module. CDC conducted additional outreach through collaboration with external partners. CMS frequently invited CDC representatives to share new NHSN-related guidance and changes, and to answer questions during webinars and weekly stakeholder calls with nursing home administrators, directors of nursing, and others. CDC explained that it also worked with nursing home chains, trade organizations, and State health departments to disseminate information to facilities. For example, the Surveillance Branch worked with internal and external partners to publish guidance that addressed frequently asked questions. One CDC official said the "comprehensive approach to getting the information out is the key to [NHSN's] success." CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 21 Ongoing nursing home challenges have led to additional enrollment-related improvements To address the significant delay in the identity verification process, CDC's Digital Services implemented two new system improvements in October 2021 to upgrade users' security access levels. One Digital Services official reported that CDC contracted with a financial credit company to expedite identity verification. The streamlined process involves SAMS directing applicant information, such as date of birth, telephone number, and Social Security number, to the contracted system for verification. The user then answers five credit history-related questions to complete the process. Users who choose not to participate in the contracted process may still use the original method. CDC further improved the enrollment process by providing a more direct way of receiving a second form of authentication. According to the Digital Services official, an individual would previously receive a physical token by mail, which allowed the user to log in to NHSN. Digital Services then introduced a self-service portal and a corresponding mobile authenticator application that provides immediate access to the system. The official reported that, after the deployment of the instant authenticator, CDC saw a significant shift from the use of physical cards to use of the mobile authenticator. CDC has collaborated with CMS and other stakeholders to reduce data variables no longer deemed necessary and further ease the reporting burden on facilities While nursing homes may benefit from reduced reporting, the potential removal of data variables posed challenges to CDC. A CDC official explained that it was more difficult to remove than to add pandemic-related fields because removing certain elements could conflict with CMS reporting requirements. CDC involved input from CMS and trade organizations in decisions about removing data variables throughout the pandemic. CDC reported that, toward the end of 2021, it began receiving and reviewing feedback from stakeholders that confirmed that certain data elements were no longer necessary. One official stated that as COVID-19 cases decreased, CDC received more feedback from nursing homes requesting that it reduce the number of questions. Facilities were more receptive about the need for data collection during increases in COVID-19 case counts, but were in "crisis mode with providing care." CDC has continued to collaborate with CMS and stakeholders to determine what data fields to change or remove in NHSN, while still fulfilling CMS reporting requirements. For example, CDC removed the Ventilator Capacity and Supplies Pathway in July 2021. According to one official, CDC began the process of eliminating variables related to ventilators after facilities and other stakeholders communicated that the information was no longer necessary to make resource decisions. CDC reviewed data to find that CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 22 nursing home use of ventilators had decreased, and removed the pathway after CMS did not contest its elimination. CDC has continued to reduce the amount of data nursing homes must report to the COVID-19 Module. As described on page 8, in November 2021, nursing homes were required to report up to 132 data points to NHSN (including questions to elaborate on selected response options). By August 2023, that number dropped to 37 data points. Nursing home experiences with reporting have improved since May 2020, though issues remain for both facilities and CDC As of June 2022, nursing homes noted that the process for reporting COVID-19 data has improved. However, some facilities still reported experiencing issues with reporting and difficulties receiving timely support from the NHSN Help Desk. Facilities reported that CDC improvements have helped nursing homes reporting COVID-19 data to NHSN Survey results indicated that CDC actions have led to improvements in the ability of nursing homes to submit data to the COVID-19 Module. Most facilities (84 percent) noted that CDC's communication regarding updates made to NHSN has improved. Facilities also recognized that guidance has improved, with 83 percent reporting that guidance on how to report data to NHSN had improved, and 80 percent reporting that guidance on what to report had improved. (See Exhibit 4 on the next page.) One facility said that miscommunication has become less of a concern because CDC guidance has become more clear. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 23 Exhibit 4: Nursing homes reported that aspects of the NHSN reporting process have improved, as of June 2022 (n=171). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. Many nursing homes no longer experience challenges with the NHSN reporting process that existed earlier in the pandemic. A number of nursing homes identified several previous issues with reporting that they no longer experienced as of June 2022. Challenges that nursing homes reported previously experiencing at a higher rate included confusion about how to report (37 percent) and technical difficulties with reporting (35 percent). Further, some facilities (16 percent) reported having past challenges with limited time or staff available to report. (See Exhibit 5 on the next page.) A corporate representative for one facility explained that clinical staff manage multiple tasks, such as conducting COVID-19 tests and maintaining supplies, and data submission added to these existing time commitments. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 24 Exhibit 5: Nursing homes reported past and current challenges with reporting to NHSN, as of June 2022 (n=171). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. Note: The total percentages for some of the challenges above do not sum to 100 pecent because of rounding. CDC reported that nursing homes had initial concerns about how to interpret certain data elements, such as which fields were requesting incident or cumulative information, and what to do with data for former residents no longer at the facilities. The agency noted a continued need to facilitate nursing home reporting to NHSN. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 25 Despite CDC actions, ongoing nursing home challenges suggest a need for further improvements to the NHSN system and processes Some nursing homes identified multiple challenges with reporting to NHSN, which were ongoing as of June 2022. For example, 36 percent of facilities reported that having limited time or staff available made it difficult to submit data, while 13 percent reported being uncertain about what data they were required to submit to NHSN. (See Exhibit 5 on the preceding page.) Some nursing homes have had worsening experiences reporting to NHSN over the course of the pandemic. For example, 22 percent of facilities reported that receiving error messages when attempting to submit data had worsened, and 9 percent reported that CDC communication regarding updates to NHSN had worsened. (See Exhibit 6.) A corporate representative for one nursing home described how the time commitment for reporting was a burden that increased over time, causing the organization to dedicate three full-time employees to handling COVID-19 reporting across their facilities. The representative also noted that the organization's internal processes for collecting and submitting required data were hampered by updates to NHSN because there was a lack of information in advance about the changes. Exhibit 6: Nursing homes reported that aspects of the NHSN reporting process have worsened, as of June 2022 (n=171). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 26 Nursing homes faced difficulties receiving timely and useful assistance from the NHSN Help Desk, which did not keep up with user support requests and communicated only via email Nursing homes reported that NHSN Help Desk responses were often not prompt or useful. Facilities reported reaching out to the NHSN Help Desk for support during the pandemic. As of June 2022, 85 percent of facilities had sought help from the Help Desk. The majority of these nursing homes (63 percent) reported having issues with the responses to their inquiries. Nursing homes reported that assistance was delayed (33 percent); not useful (8 percent); or both delayed and not useful (19 percent). A small number of nursing homes (3 percent) reported not receiving a response from the Help Desk. (See Exhibit 7.) Exhibit 7: Nursing homes reported difficulties receiving timely and useful assistance from the NHSN Help Desk, as of June 2022 (n=145). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. The NHSN Help Desk only supported users via email, without alternatives such as a phone line or live chat support. Several nursing home representatives noted that this made interactions particularly cumbersome and frustrating. One facility representative said they submitted multiple questions to the Help Desk, but never received responses that specifically addressed their concerns. Rather, they always received generic responses that referenced the NHSN webpage for frequently asked questions. An administrator from another nursing home reported receiving delayed assistance from the NHSN Help Desk, which resulted in the facility missing a reporting deadline and being fined. A corporate representative for one nursing home noted reaching out to a Quality Improvement Organization for assistance instead of continuing to contact the Help Desk.71 The representative explained that Help Desk CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 27 responses did not address the facility's questions, and reaching out to the Help Desk was confusing in general. The NHSN Help Desk had ongoing difficulty addressing its backlog of user requests, despite using strategies to streamline processes. Despite CDC efforts, the NHSN Help Desk continues to experience a backlog of user support requests. CDC reported that it detailed additional staff and pulled members from other teams to help respond to user inquiries. Still, the Help Desk experienced challenges in offsetting the continuous influx of tickets brought about by the substantial increase in total users. One official explained that the NHSN Help Desk received 300 to 500 new tickets daily, which were "physically impossible" to individually triage on the same day. In addition, the NHSN Help Desk often needed more information than what users provided in their email inquiries to deliver thorough responses. The need for back-and-forth correspondence, in addition to the large quantity of tickets, contributed to delay of responses. One CDC official added that staff turnover in nursing homes, which resulted in new users, was another cause of the continuing flow of Help Desk tickets. The official also reported that the Surveillance Branch ultimately did not have the necessary resources to fully respond to facilities requesting assistance at this scale. A different official said that, as of May 2022, the NHSN Help Desk still had a backlog of unanswered and untriaged tickets, but that the backlog was not as significant as it was in 2020. According to CDC, the NHSN Help Desk used various strategies to improve efficiency. The agency reported that the Help Desk added internal training for user support specialists. In addition, Help Desk staff developed a bank of "canned responses," or copied and pasted template responses, to common inquiries. The NHSN Help Desk also created a new email address specifically for enrollment-related questions, and encouraged users to include specific subject lines in their emails to expedite the triage of inquiries to specific NHSN subject-matter experts. CDC has made efforts to ensure data quality, but nursing homes reporting to the COVID-19 Module expressed concerns CDC established QA criteria to address data issues identified during the early stage of the required COVID-19 nursing home reporting. The agency conducted outreach to some facilities with notable data concerns and built visual alerts for QA flags into the system. Despite CDC efforts, some facilities reported not feeling confident about the completeness or accuracy of the nursing home COVID-19 data in NHSN. The agency identified data quality issues and built QA checks into its processes CDC flagged potential data quality problems early in its data collection. According to agency officials, initial reviews of COVID-19 data revealed that many nursing homes CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 28 reported cumulative data instead of incident-level data. CDC analyzed data from the COVID-19 Module in May and June 2020 to identify patterns that may suggest data entry errors, such as repeated or cumulative values, in certain variables related to COVID-19 cases and deaths. CDC then established QA criteria to identify outliers that indicate potential data quality issues. CDC began contacting nursing homes in June 2020 in response to the initial data quality issues. The agency stated that the Surveillance Branch produced monthly reports with an emphasis on information related to COVID-19 cases, deaths, and other key fields. If a facility's data exceeded the outlier threshold, Surveillance Branch staff emailed or called the nursing home, or notified the relevant State health department. l Once in contact with the facility, the agency staff members verified the information or helped the nursing home user correct it. According to one CDC official, the frequency of outreach to nursing homes has declined over time. CDC also built visual QA flags into the COVID-19 Module to identify and reduce data errors. If submitted data activate the QA flags, system alerts visually notify the user of possible data entry issues. The user can then click on the alert to navigate to the issue and fix the problem. CDC reported that the Surveillance Branch now takes a two- pronged approach to improve data quality: visual alerts in NHSN and direct outreach, when necessary. CMS's QA checks provide another opportunity to identify problems with data reported to the COVID-19 Module. CDC reported that NHSN shares nursing home COVID-19 data with CMS at the end of each reporting week, after which CMS conducts its own QA check on the data. CMS removes a nursing home's data from its publicly available nursing home datasets if the data failed its QA checks. CMS is able to take enforcement actions, including fines, if it identifies that a facility has not met its weekly reporting obligations. Nursing home challenges with reporting may result in inaccurate COVID-19 data reported to NHSN More than one in four facilities reported not being confident that the data nursing homes have submitted to NHSN are complete and accurate. One administrator stated that data may be omitted in instances of facility staff turnover, after which the new NHSN Administrator does not have access to previous records used to submit data to the COVID-19 Module. "I can almost guarantee you, as complicated as this data is, there are facilities that are not reporting correctly." – Nursing home corporate representative _____________________________________________________________________________________________________________________________ lCDC does not have the authority to take punitive action against a nursing home that does not meet its weekly reporting obligation. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 29 A corporate representative for another facility expressed having no confidence that the data are accurate, because the definitions for the data variables are unclear and leave room for varying interpretations. Regarding COVID-19 vaccinations, one nursing home representative said it was difficult to have complete confidence with staff vaccination counts and rates when turnover leads to complicated week-to-week changes in data. CDC plans to continue improving NHSN for nursing home reporting and is considering how to leverage national enrollment for voluntary reporting on other health events As noted, CMS will continue to require nursing homes to report a full range of COVID-19 data through December 2024, while the vaccination-related data mandate will be ongoing. This means that required weekly reporting by nursing homes to NHSN will continue. CDC officials reported considering how the agency might leverage this national enrollment and reporting to track other health events. In May 2023, CDC reported that it has been focused on current reporting and related changes to the system, and has not yet planned an approach for encouraging voluntary reporting after some requirements expire. One official noted that CDC intends to keep all portions of the COVID-19 Module accessible to nursing homes after requirements expire. Additionally, the agency intends to encourage voluntary reporting to NHSN. There is precedent for voluntary reporting by nursing homes to NHSN. Prior to the COVID-19 pandemic, more than 3,000 nursing homes were enrolled in NHSN to voluntarily report data on HAIs and infection prevention measures. One CDC official noted that during the pandemic, a third of nursing homes (about 5,000) reported data on individual COVID-19 tests for optional data fields. Another official said that it is difficult to accurately predict how many nursing homes will voluntarily report COVID-19 data after the requirements expire. CDC plans to continue modernization of NHSN data collection, such as automating data input CDC reported that it is investing significant resources to modernize NHSN as part of a larger data initiative within the agency. m One official stated that it is essential to continue the "modernization of the [information technology] structure" within NHSN, with the goal of making the system more automated and less of a burden on facilities. For example, according to CDC officials, the agency has considered the possibility of _____________________________________________________________________________________________________________________________ m In July 2023, CDC clarified to OIG that continued modernization and enhancement of NHSN were contingent on the agency receiving funds detailed in its 2024 budget request to Congress. See HHS, CDC, Fiscal Year 2024 Justification of Estimates for Appropriations. Accessed at https://www.cdc.gov/budget/documents/fy2024/FY-2024-CDC-congressional-justification.pdf on August 15, 2023. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 30 developing an interaction between NHSN and facility electronic health records that would input certain information directly into NHSN. As a part of pursuing its modernization objectives, CDC is looking at ways to improve data collection efforts in the future, in part by reflecting on its experience using NHSN during its pandemic response. One official noted that automated improvements to SAMS, such as streamlined identity verification during enrollment, have saved time and effort granting facilities access to the system. CDC reported that its teams have continued to rethink the best ways to be "more proactive and ready to help" nursing homes interacting with NHSN for ongoing required and voluntary reporting. Facilities suggested several ways CDC could better facilitate nursing home reporting to NHSN. Nursing homes noted that CDC should be more responsive and easier to reach for assistance. More than one-third of facilities (58 of 171) reported that CDC being more responsive to support tickets would be the most significant way to improve NHSN support. (See Exhibit 8.) One nursing home said that a phone line for the NHSN Help Desk is necessary for providing more immediate assistance. Another facility agreed that telephone support should be offered and added that it would be particularly helpful with the enrollment process. Exhibit 8: Nursing homes reported actions CDC could take to better facilitate nursing home reporting to NHSN (n=171). Source: OIG analysis of survey responses from 171 nursing homes, November 2022. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 31 CDC reported that it is considering how to leverage national nursing home enrollment in NHSN to encourage participation in reporting for other health care outcomes One agency official reported that multiple stakeholders have expressed the desire to maintain national nursing home reporting after the public health emergency. The agency has communicated with CMS about the value of nursing home reporting and the use of NHSN for other public health surveillance. CDC reported that it continues to collaborate with CMS about COVID-19 data requirements and how NHSN data could be used to inform other public health and nursing home quality and safety efforts, such as CMS quality measures for nursing homes. A CDC official said that NHSN has "clearly demonstrated it is the platform" to be used for these data collection efforts, and that nursing home reporting to the COVID-19 Module could serve as a foundation for collecting data related to various health care issues. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Findings | 32 CONCLUSION AND RECOMMENDATIONS NHSN has been a critical data collection tool for the national COVID-19 pandemic response in nursing homes. Following initial challenges with the influx of nursing home enrollment, CDC made improvements to the enrollment and reporting processes. CDC collaborated with external partners, such as CMS, and other stakeholders to disseminate guidance, communicate with nursing homes, and implement important changes to the COVID-19 Module. Facilities reported that some challenges were resolved, but that other difficulties are ongoing. Over time, CDC has reduced the number of variables needed to meet requirements. Beginning in January 2025, required variables related to infections, deaths, and staffing will expire. Nursing homes will continue to be required to report COVID-19 vaccination data to NHSN, and may voluntarily report other variables. CDC efforts have resulted in a better reporting experience, but more improvements are needed to facilitate nursing home reporting and ensure data quality. These improvements will also facilitate CDC efforts to leverage national nursing home enrollment for potential ongoing and future use of NHSN. We recommend that CDC: Improve the user support the NHSN Help Desk provides to nursing homes CDC officials recognized that the NHSN Help Desk faced challenges due to the influx of facilities requesting extensive support. The Help Desk, which is only accessible via email, was quickly overwhelmed with user support requests, but did not see a proportional increase in staff to meet the higher demand. Despite agencywide efforts, such as the deployment of the "click squad," the NHSN Help Desk has struggled to meet the needs of nursing homes reporting to the system. CDC should add communication channels to provide direct assistance to NHSN users. This should include adding telephone or live chat support functions so nursing homes can have questions answered in real time and provide feedback about their experiences enrolling in and reporting to the system. Providing an option for NHSN users to directly communicate with a user support specialist may promote more accurate, timely, and complete reporting, and build on the relationship between the agency and NHSN users. CDC should also improve its ability to increase its Help Desk staffing during periodic influxes of requests, such as when reporting needs or guidance changes. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Conclusion and Recommendations | 33 Take further steps to ensure the quality of nursing home reporting of COVID-19 data to NHSN To address nursing home concerns about the accuracy of COVID-19 data reported to NHSN, CDC should take further steps to provide guidance and technical assistance to facilities, and to ensure appropriate QA processes. The agency should work toward addressing the root causes of data quality issues by reviewing its overall approach to guidance and support, to best determine what efforts have been beneficial to fostering quality data reporting. For example, CDC could issue more explicit guidance on data definitions to reduce variations in nursing homes' interpretations, and offer technical assistance related to maintaining data reporting during times of staff turnover. For its participation in webinars and trainings, CDC should ensure that it provides detailed and practical responses to the array of nursing home questions it may encounter. Consider how quality assurance checks can be enhanced to ensure data accuracy, as appropriate To support quality data collection efforts ongoing and in the future, CDC should assess how it can enhance its QA checks for the full range of COVID-19 data it will collect from nursing homes. Data that nursing homes enter into NHSN is self-reported. It is possible that facilities submit inaccurate information that is not detected by automated review processes currently built into the system. Therefore, CDC should consider how it can improve existing QA checks, as well as determining what additional QA processes it can use to better ensure that NHSN has accurate data from facilities. CDC could also apply these ideas to promote data accuracy and quality in any future expansions of its data collection from nursing homes through NHSN. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Conclusion and Recommendations | 34 AGENCY COMMENTS AND OIG RESPONSE CDC partially concurred with the first recommendation and concurred with the second and third recommendations, and reported its planned actions to implement the recommendations. CDC partially concurred with the first recommendation to improve the user support the NHSN Help Desk provides to nursing homes. CDC reported that it has made improvements to the Help Desk and now offers more ways for facilities to receive support through a new customer service portal. CDC stated that in the new portal, inquiries from NHSN users are sent directly to the appropriate support specialist or subject-matter expert, resulting in faster responses. CDC also noted that it has increased staffing and relevant trainings for the Help Desk, and that there is no longer a backlog of support requests from nursing homes. OIG appreciates CDC's efforts to improve the NHSN Help Desk support for its users, and acknowledges CDC's increased staffing for the Help Desk. However, OIG continues to recommend that CDC also prioritize implementing telephone or live chat functionality for its Help Desk, when resources are available, to ensure that facilities have the option to obtain real- time assistance. Providing direct user communication with the Help Desk can help promote accurate, timely, and complete reporting. OIG also continues to recommend that CDC ensure its ability to dedicate or reallocate staff, as necessary, in case an influx of requests occurs again. CDC concurred with our second recommendation to take further steps to ensure the quality of nursing home reporting of COVID-19 data to NHSN. CDC stated that it will expand its efforts to provide training materials and tools to NHSN users and health care facilities. CDC reported that it has taken a multi-pronged approach to ensure accurate vaccination status data reporting, including providing training on NHSN for LTCFs and conducting real-time outlier analysis that has led to the agency contacting more than 8,000 facilities about their data. OIG appreciates CDC's efforts to improve the quality of nursing home reporting to NHSN. We believe that CDC's planned actions, when completed, will fulfill this recommendation. CDC also concurred with our third recommendation to consider how quality assurance checks can be enhanced to ensure data accuracy, as appropriate. CDC stated that it will review its processes to identify opportunities to improve its data quality checks, user education, and outreach. CDC also noted that incorporating active quality checks, such as data validation, would require additional resources and staff. OIG appreciates CDC's commitment to identify opportunities for improvement. We look forward to the results of CDC's review and learning more about CDC's improvement plans in its final management decision. For the full text of CDC's comments, see Appendix E. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Agency Comments and OIG Response | 35 APPENDICES Appendix A: Estimates, Confidence Intervals, and Key Statistics for Nursing Home Survey Results The exhibits listed below provide key statistics to support our projected results for each nursing home survey question used in the findings of this report. The estimates are based on a sample of 171 nursing homes that reported required COVID-19 data to the NHSN LTCF COVID-19 Module at least once between May 24 and December 26, 2021. Exhibits A-1 through A-3 (n=117) and Exhibit A-7 (n=145) use subsets of nursing homes resulting from respondents selecting specific answers that filtered survey questions. Exhibit A-1: Nursing homes that reported difficulty with at least one aspect of enrollment in their responses to "How easy or difficult were the following aspects of enrolling your facility in NHSN?" (n=117) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Any enrollment difficulty 101 86.32% 78.74% 91.98% Exhibit A-2: Nursing home responses to "How easy or difficult were the following aspects of enrolling your facility in NHSN?" (n=117) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Finding guidance and resources Very easy 14 11.97% 6.70% 19.26% Somewhat easy 48 41.03% 32.02% 50.50% Somewhat difficult 33 28.21% 20.28% 37.27% Very difficult 13 11.11% 6.05% 18.25% I don't know 9 7.69% 3.58% 14.10% Registering users Very easy 12 10.26% 5.41% 17.23% Somewhat easy 26 22.22% 15.06% 30.84% Somewhat difficult 52 44.44% 35.26% 53.92% Very difficult 26 22.22% 15.06% 30.84% I don't know 1 0.85% 0.02% 4.67% CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 36 Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Upgrading SAMS from Level 1 to Level 3 Very easy 6 5.13% 1.90% 10.83% Somewhat easy 19 16.24% 10.07% 24.19% Somewhat difficult 38 32.48% 24.11% 41.76% Very difficult 46 39.32% 30.41% 48.77% I don't know 8 6.84% 3.00% 13.03% Receiving electronic communication Very easy 33 28.21% 20.28% 37.27% Somewhat easy 46 39.32% 30.41% 48.77% Somewhat difficult 24 20.51% 13.61% 28.97% Very difficult 13 11.11% 6.05% 18.25% I don't know 1 0.85% 0.02% 4.67% Receiving technical assistance Very easy 14 11.97% 6.70% 19.26% Somewhat easy 31 26.50% 18.77% 35.45% Somewhat difficult 30 25.64% 18.02% 34.54% Very difficult 36 30.77% 22.57% 39.97% I don't know 6 5.13% 1.90% 10.83% Exhibit A-3: Nursing home responses to "What actions did your facility take to address difficulties enrolling in NHSN?" (n=117) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Held staff training on 14 11.97% 6.70% 19.26% enrolling in NHSN Acquired additional 2 1.71% 0.21% 6.04% technology Requested assistance 57 48.72% 39.37% 58.13% from CDC Requested assistance 35 29.91% 21.80% 39.07% from CMS Requested assistance from nongovernment 23 19.66% 12.89% 28.02% organization Requested assistance CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 37 Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound from your facility chain or 45 38.46% 29.62% 47.91% corporate ownership I do not know 2 1.71% 0.21% 6.04% Our facility did not take 9 7.69% 3.58% 14.10% any action Receiving technical 3 2.56% 0.53% 7.31% assistance Exhibit A-4: Nursing home responses to "To what extent have the following aspects of reporting to NHSN become better or worse since your facility began using the system?" (n=171) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Guidance on how to report Better 142 83.04% 76.56% 88.34% Somewhat better 97 56.73% 48.94% 64.27% Much better 45 26.32% 19.89% 33.58% Worse 13 7.60% 4.11% 12.65% Somewhat worse 11 6.43% 3.25% 11.22% Much worse 2 1.17% 0.14% 4.16% I don't know 16 9.36% 5.44% 14.75% Guidance on what to report Better 137 80.12% 73.34% 85.82% Somewhat better 93 54.39% 46.61% 62.01% Much better 44 25.73% 19.36% 32.96% Worse 20 11.70% 7.29% 17.48% Somewhat worse 17 9.94% 5.90% 15.44% Much worse 3 1.75% 0.36% 5.04% I don't know 14 8.19% 4.55% 13.36% CDC communication of updates Better 143 83.63% 77.21% 88.84% Somewhat better 87 50.88% 43.13% 58.59% Much better 56 32.75% 25.78% 40.33% Worse 16 9.36% 5.44% 14.75% Somewhat worse 11 6.43% 3.25% 11.22% CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 38 Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Much worse 5 2.92% 0.96% 6.69% I don't know 12 7.02% 3.68% 11.94% Adding or changing users to report data Better 101 59.06% 51.30% 66.51% Somewhat better 74 43.27% 35.73% 51.06% Much better 27 15.79% 10.67% 22.14% Worse 32 18.71% 13.17% 25.38% Somewhat worse 23 13.45% 8.72% 19.50% Much worse 9 5.26% 2.43% 9.76% I don't know 38 22.22% 16.23% 29.20% Submitting data without error messages Better 121 70.76% 63.33% 77.45% Somewhat better 81 47.37% 39.70% 55.13% Much better 40 23.39% 17.27% 30.46% Worse 37 21.64% 15.72% 28.57% Somewhat worse 25 14.62% 9.69% 20.82% Much worse 12 7.02% 3.68% 11.94% I don't know 13 7.60% 4.11% 12.65% Ability to view and edit data Better 138 80.70% 73.98% 86.33% Somewhat better 79 46.20% 38.56% 53.97% Much better 59 34.50% 27.41% 42.14% Worse 12 7.02% 3.68% 11.94% Somewhat worse 11 6.43% 3.25% 11.22% Much worse 1 0.58% 0.01% 3.22% I don't know 21 12.28% 7.77% 18.16% CDC communication on errors and issues Better 120 70.18% 62.72% 76.92% Somewhat better 80 46.78% 39.13% 54.55% Much better 40 23.39% 17.27% 30.46% Worse 24 14.04% 9.20% 20.16% Somewhat worse 17 9.94% 5.90% 15.44% Much worse 7 4.09% 1.66% 8.25% I don't know 27 15.79% 10.67% 22.14% CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 39 Exhibit A-5: Nursing home responses to "Has your facility experienced any of the following challenges with reporting data to NHSN?" (n=171) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Confusion about how to report Previous challenge 64 37.43% 30.16% 45.14% Current challenge 19 11.11% 6.82% 16.81% No challenge 87 50.88% 43.13% 58.59% I don't know 1 0.58% 0.01% 3.22% Technical issues Previous challenge 60 35.09% 27.96% 42.74% Current challenge 45 26.32% 19.89% 33.58% No challenge 64 37.43% 30.16% 45.14% I don't know 2 1.17% 0.14% 4.16% System not accepting data entries Previous challenge 58 33.92% 26.87% 41.54% Current challenge 38 22.22% 16.23% 29.20% No challenge 73 42.69% 35.17% 50.47% I don't know 2 1.17% 0.14% 4.16% Difficulty receiving assistance Previous challenge 53 30.99% 24.16% 38.51% Current challenge 34 19.88% 14.18% 26.66% No challenge 80 46.78% 39.13% 54.55% I don't know 4 2.34% 0.64% 5.88% Unclear or complex data fields Previous challenge 52 30.41% 23.62% 37.89% Current challenge 40 23.39% 17.27% 30.46% No challenge 78 45.61% 37.99% 53.39% I don't know 1 0.58% 0.01% 3.22% Uncertainty on required data Previous challenge 52 30.41% 23.62% 37.89% Current challenge 23 13.45% 8.72% 19.50% No challenge 94 54.97% 47.19% 62.58% I don't know 2 1.17% 0.14% 4.16% Staff turnover CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 40 Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Previous challenge 44 25.73% 19.36% 32.96% Current challenge 27 15.79% 10.67% 22.14% No challenge 96 56.14% 48.36% 63.70% I don't know 4 2.34% 0.64% 5.88% Lack of communication from CDC about how to fix data errors Previous challenge 39 22.81% 16.75% 29.83% Current challenge 26 15.20% 10.18% 21.48% No challenge 92 53.80% 46.03% 61.44% I don't know 14 8.19% 4.55% 13.36% Difficulty collecting data Previous challenge 36 21.05% 15.20% 27.93% Current challenge 42 24.56% 18.31% 31.72% No challenge 91 53.22% 45.45% 60.87% I don't know 2 1.17% 0.14% 4.16% Limited time or staff availability Previous challenge 28 16.37% 11.16% 22.79% Current challenge 62 36.26% 29.06% 43.94% No challenge 79 46.20% 38.56% 53.97% I don't know 2 1.17% 0.14% 4.16% Exhibit A-6: Nursing home responses to "CDC has an NHSN User Support Help Desk (NHSN Help Desk) to assist facilities that use the system. Has your facility reached out to the NHSN Help Desk?" (n=171) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Yes, reached out to 145 84.80% 78.52% 89.82% Help Desk Exhibit A-7: Nursing home responses to "Overall, how responsive was the NHSN Help Desk to your facility's request(s) for assistance?" (n=145) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Prompt and useful 50 34.48% 26.79% 42.82% Not helpful 91 62.76% 54.35% 70.64% CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 41 Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Prompt, but not useful 11 7.59% 3.85% 13.17% Delayed, but still useful 48 33.10% 25.52% 41.40% Delayed and not useful 28 19.31% 13.23% 26.69% Facility did not receive 4 2.76% 0.76% 6.91% a response I don't know 4 2.76% 0.76% 6.91% Exhibit A-8: Nursing home responses to "In your opinion, are you confident that nursing homes have submitted complete and accurate data to NHSN?" (n=171) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound No 49 28.65% 22.01% 36.06% Yes 100 58.48% 50.71% 65.95% I don't know 22 12.87% 8.24% 18.83% Exhibit A-9: Nursing home responses to "How can CDC better support your facility's ability to report COVID-19 data to NHSN?" (n=171) Number Percentage 95-Percent Confidence Interval Response Description in Sample Estimate Lower Bound Upper Bound Be more responsive to 58 33.92% 26.87% 41.54% requests for assistance Provide better written 56 32.75% 25.78% 40.33% instructions Provide more hands-on 49 28.65% 22.01% 36.06% or interactive training Improve communication 42 24.56% 18.31% 31.72% of NHSN updates Request more direct 29 16.96% 11.66% 23.44% feedback from facilities CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix A | 42 Appendix B: Changes to the LTCF COVID-19 Module In April 2020, the LTCF COVID-19 Module consisted only of the Surveillance Reporting Pathways. 72 Later, the module expanded to include the Point-of-Care (POC) Test Reporting Tool, in October 2020; the COVID-19 Vaccinations Module, in December 2020; and the Therapeutics Pathway, in February 2021. 73, 74, 75 CDC removed three pathways from the Surveillance Reporting Pathways: Ventilator Capacity and Supplies in July 2021, Supplies and PPE in May 2022, and Therapeutics in June 2023. 76, 77, 78 (See Exhibit B-1.) Exhibit B-1: Timeline of major CDC updates to the LTCF COVID-19 Module Sources: CDC, COVID-19 Module Enrollment Guidance for LTCFs, May 2020. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/ltcf-covid19-module-508.pdf on March 7, 2023. CDC, COVID-19 Point of Care (POC) Test Result Reporting Tool Frequently Asked Questions, March 2022. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/ltcf-poc-faq-508.pdf on March 7, 2023. CDC, HCP & Resident COVID-19 Vaccinations. Accessed at https://www.cdc.gov/nhsn/ltc/weekly-covid-vac/index.html on March 7, 2023. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix B | 43 Appendix C: Step-by-Step Enrollment Process for NHSN Users The following instruction is from the "5-Step Enrollment for Long-term Care Facilities" portion of the CDC NHSN webpage on January 27, 2023. 79 Step 1: Enrollment Preparation 1. Review required LTCF Enrollment Guidance. Note: The LTCF Enrollment Training slides serve as a detailed guide through the enrollment process. 2. Check trusted websites and spam blocker settings. • Please use the supported browsers, Microsoft Edge or Google Chrome. Note: Microsoft Windows Users: Microsoft is moving away from supporting Internet Explorer 11 (IE) in 2021. Do not use Internet Explorer when accessing NHSN. • Add cdc.gov and verisign.com to your list of trusted websites and permit pop-ups for these sites in the browser. • Check spam-blocker settings to allow emails from NHSN@cdc.gov, SAMS- NO-REPLY@cdc.gov. • For information regarding System requirements, visit: FAQs About NHSN | NHSN | CDC. Time to complete Step 1: 40 minutes Step 2: Register Facility with NHSN Read and agree to the NHSN Rules of Behavior. You will then be redirected to electronically register your facility with NHSN. Note: The NHSN Facility Contact form may be used to manually collect required registration information prior to electronically entering the data into NHSN. Time to complete Step 2: 10 minutes CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix C | 44 Step 3a: Register with SAMS Partner Portal You will receive an email from the "U.S. Centers for Disease Control: SAMS Partner Portal – Invitation to Register." It will include a web link to the SAMS registration page along with your assigned SAMS Partner Portal User ID (email address) and a temporary password. Please click on the link and enter the SAMS Partner Portal User ID and temporary password when prompted. Then follow the onscreen instructions to register and accept the SAMS Rules of Behavior. Note: After being invited to SAMS, you have 30 days to complete the registration process. Time to complete Step 3a: 15 minutes Step 3b: Identity Proofing Verification (if applicable) After successful online registration, you will receive an "Identity Verification Request" email from SAMS with instructions and a request to provide proof of your identity. SAMS supports two options for validating your identity using the New online preferred method Experian or the current secured document submission process. * Please follow the instructions carefully for your selected identity verification option, to ensure timely processing. Time to complete Step 3b: 35 minutes Step 4: Access Approval and Account Activation Once your information is received and approved, you will receive your approval by email from sams-no-reply@cdc.gov. The subject will be "U.S. Centers for Disease Control: SAMS Partner Portal – SAMS Activity Authorization." This email will contain web links to access the SAMS Partner Portal and application. The NHSN application requires users to be strongly authenticated. This means users must have an additional 'token', along with their password, to log in. The first option is a soft token that requires the installation of an Entrust Authenticator application on your phone, tablet, or computer. The second option is a hard token which is a physical Entrust grid card mailed to your home address. SAMS will communicate this authentication requirement to you in this email along with instructions on how to complete the setup after account activation. Note: Your SAMS grid card should be delivered to your home address via U.S. mail within 2 weeks after you receive your SAMS approval email. If you do not receive your SAMS grid card within two weeks, contact samshelp@cdc.gov for assistance. Time to complete Step 4: 40 minutes CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix C | 45 Step 5a: Complete NHSN Enrollment After setting up your credential device (via token or receiving your SAMS grid card), login to SAMS. Under the SAMS Multi-factor Login picture, click Login. On the SAMS home page, select 'NHSN Enrollment' to complete electronic enrollment into the Long Term Care Facility Component. Note: Facility registration information may be manually collected using the NHSN Facility Contact and Annual Facility Survey forms prior to electronically entering the data into NHSN. Time to complete Step 5a: 40 minutes Shortly after successfully submitting the forms, the Facility Administrator and Primary Contact will receive a "NHSN Facility Enrollment Submitted" email. Step 5b: Accept "NHSN Agreement to Participate and Consent" Either the component primary contact or NHSN facility administrator must login to SAMS, select Long-term Care Facility Component and review the "Agreement to Participate and Consent." Check the box to 'Accept' next to the appropriate contact name and then 'Submit.' An email will be issued confirming this action. Note: The consent form must be accepted within 60 days or the facility will be withdrawn. Time to complete Step 5b: 5 minutes Congratulations, enrollment is complete! Complete NHSN Long-term Care Facility specific training if not previously completed. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix C | 46 Appendix D: SAMS Registration and Identity Verification Upon receiving the invitation to SAMS, applicants complete the SAMS registration process by validating their legal name and providing their organization and contact information. 80 The invitation is only usable for a single registration and expires after 30 days. If the invitation is lost or expired, the applicant must request a replacement. Following registration, some applicants undergo an identity verification process. Applicants verify their identity through a document review, or a credit history review by a data analytics and financial credit company contracted by CDC. 81 For the document review, the applicant must submit a form and identity proofing documents to SAMS. The form must be reviewed by a notary or other trusted third parties prior to submission. Following SAMS registration, a CDC program administrator approves the user's access to SAMS and the application. 82 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix D | 47 Appendix E: Agency Comments CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix E | 48 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix E | 49 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix E | 50 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix E | 51 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Appendix E | 52 ACKNOWLEDGMENTS AND CONTACT Acknowledgments John Van Der Schans served as the project leader for this study. Others in the Office of Evaluation and Inspections Dallas Regional Office who conducted the study include Marshall Allen and Sarah Lee. Office of Evaluation and Inspections headquarters staff who provided support include Rob Gibbons and Michael Novello. This report was prepared under the direction of Petra Nealy, Regional Inspector General for Evaluation and Inspections in the Dallas Regional Office. Contact To obtain additional information concerning this report, contact the Office of Public Affairs at Public.Affairs@oig.hhs.gov. OIG reports and other information can be found on the OIG website at oig.hhs.gov. Office of Inspector General U.S. Department of Health and Human Services 330 Independence Avenue, SW Washington, DC 20201 CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Acknowledgments and Contact | 53 ABOUT THE OFFICE OF INSPECTOR GENERAL Office of Inspector General https://oig.hhs.gov The mission of the Office of Inspector General (OIG) is to provide objective oversight to promote the economy, efficiency, effectiveness, and integrity of the Department of Health and Human Services (HHS) programs, as well as the health and welfare of the people they serve. Established by Public Law No. 95-452, as amended, OIG carries out its mission through audits, investigations, and evaluations conducted by the following operating components: The Office of Audit Services. OAS provides auditing services for HHS, either by conducting audits with its own audit resources or by overseeing audit work done by others. The audits examine the performance of HHS programs, funding recipients, and contractors in carrying out their respective responsibilities and provide independent assessments of HHS programs and operations to reduce waste, abuse, and mismanagement. The Office of Evaluation and Inspections. OEI's national evaluations provide HHS, Congress, and the public with timely, useful, and reliable information on significant issues. To promote impact, OEI reports also provide practical recommendations for improving program operations. The Office of Investigations. OI's criminal, civil, and administrative investigations of fraud and misconduct related to HHS programs and operations often lead to criminal convictions, administrative sanctions, and civil monetary penalties. OI's nationwide network of investigators collaborates with the Department of Justice and other Federal, State, and local law enforcement authorities. OI works with public health entities to minimize adverse patient impacts following enforcement operations. OI also provides security and protection for the Secretary and other senior HHS officials. The Office of Counsel to the Inspector General. OCIG provides legal advice to OIG on HHS programs and OIG's internal operations. The law office also imposes exclusions and civil monetary penalties, monitors Corporate Integrity Agreements, and represents HHS's interests in False Claims Act cases. In addition, OCIG publishes advisory opinions, compliance program guidance documents, fraud alerts, and other resources regarding compliance considerations, the anti-kickback statute, and other OIG enforcement authorities. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 About OIG | 54 ENDNOTES 1Lenny Bernstein and Alauna Safarpour, "Mask shortage for most health-care workers extended into May, Post-Ipsos poll shows," The Washington Post, May 20, 2020. Accessed at https://www.washingtonpost.com/health/mask-shortage-for-most-health-care-workers-extended-into-may-post-ipsos-poll- shows/2020/05/20/1ddbe588-9a21-11ea-ac72-3841fcc9b35f_story.html on November 2, 2021. 2Lauren Weber, "Nursing Homes Keep Losing Workers," The Wall Street Journal, August 25, 2021. Accessed at https://www.wsj.com/articles/nursing-homes-keep-losing-workers-11629898200 on November 2, 2021. 3 CMS, COVID-19 Nursing Home Data. Accessed at https://data.cms.gov/covid-19/covid-19-nursing-home-data on January 13, 2023. 4 85 Fed. Reg. 27627 (May 8, 2020); 42 CFR § 483.80. 5 CDC, "National Healthcare Safety Network (NHSN) report: Data summary for 2006 through 2008," Association for Professionals in Infection Control and Epidemiology, December 2009, p. 1. Accessed at https://www.cdc.gov/nhsn/pdfs/datastat/2009nhsnreport.pdf on November 30, 2022. 6CDC, Division of Healthcare Quality Promotion (DHQP). Accessed at https://www.cdc.gov/ncezid/dhqp/index.html on November 30, 2022. 7 CDC, FY 2022 Operating Plan, February 19, 2021, p. 1. Accessed at https://www.cdc.gov/budget/documents/fy2021/FY-2021-CDC-Operating-Plan.pdf on December 16, 2022. 8CDC, July 2020 LTCF Newsletter, July 6, 2021. Accessed at https://www.cdc.gov/nhsn/ltc/newsletters/jul20-newsletter-ltc.html on November 17, 2022. 9CDC, "Introduction to NHSN's Long Term Care Facility Components," YouTube, September 23, 2019. Accessed at https://www.youtube.com/watch?v=pFGVqyE-H-4 on November 17, 2022. 10 CDC, Long-term Care Facility Component. Accessed at https://www.cdc.gov/nhsn/ltc/index.html on November 17, 2022. 11CDC, NHSN Long-term Care Facility Component, January 2020, p. 10. Accessed at https://www.cdc.gov/nhsn/pdfs/ltc/ltcf-manual-508.pdf on December 16, 2022. 12 CDC, LTCF COVID-19 Module. Accessed at https://www.cdc.gov/nhsn/ltc/covid19/index.html on November 17, 2022. 13CDC, July 2020 LTCF Newsletter, July 6, 2021. Accessed at https://www.cdc.gov/nhsn/ltc/newsletters/jul20-newsletter-ltc.html on November 17, 2022. CDC, LTCF COVID-19 Module Enrollment. Accessed at https://www.cdc.gov/nhsn/ltc/covid19/enroll.html on 14 November 17, 2022. 15CDC, July 2020 LTCF Newsletter, July 6, 2021. Accessed at https://www.cdc.gov/nhsn/ltc/newsletters/jul20-newsletter-ltc.html on November 17, 2022. 16CDC, "LTCF COVID-19 Module: Resident Impact and Facility Capacity Surveillance Reporting Pathway," YouTube, April 2, 2021. Accessed at https://www.youtube.com/watch?v=mrGOM4-hgAo on December 13, 2022. 17CDC, "LTCF COVID-19 Module: Resident Impact and Facility Capacity Surveillance Reporting Pathway," YouTube, April 2, 2021. Accessed at https://www.youtube.com/watch?v=mrGOM4-hgAo on December 13, 2022. 18CDC, "LTCF COVID-19 Module: Therapeutics Surveillance Reporting Pathway," YouTube, April 19, 2021. Accessed at https://www.youtube.com/watch?v=ZTzRk7-3Qak on December 15, 2022. 19CDC, LTCF COVID-19 Module: Updates, July 2021. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/rel955-update-508.pdf on December 15, 2022. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Endnotes | 55 20CDC, LTCF COVID-19 Module: Surveillance Pathway Updates, May 2022. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/covid19-mod-surv-path-508.pdf on December 15, 2022. 21 CDC, COVID-19 Module Enrollment. Accessed at https://www.cdc.gov/nhsn/ltc/covid19/enroll.html on November 17, 2022. 22CDC, Long-Term Care Facility (LTCF) Enrollment in LTCF COVID-19 Module, November 2021, p. 2. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/covid19-enrollment-508.pdf on September 16, 2021. 23CDC, Long-Term Care Facility (LTCF) Enrollment in LTCF COVID-19 Module, November 2021, p. 8. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/covid19-enrollment-508.pdf on September 16, 2021. 24CDC, Enrollment in LTCF COVID-19 Module, November 2021, pp. 2-3. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/covid19-enrollment-508.pdf on November 17, 2022. 25 CDC, COVID-19 Module Enrollment. Accessed at https://www.cdc.gov/nhsn/ltc/covid19/enroll.html on November 17, 2022. 26CDC, Enrollment in LTCF COVID-19 Module, November 2021. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/covid19-enrollment-508.pdf on November 17, 2022. 27 CDC, About SAMS. Accessed at https://www.cdc.gov/nhsn/sams/about-sams.html on November 17, 2022. 28CDC, Office of the Chief Information Officer Organization Chart. Accessed at https://www.cdc.gov/od/ocio/docs/OCIO_org_chart_FY2020.pdf on December 16, 2022. 29 Comagine Health and Medicare QIN-QIO for Idaho, Nevada, New Mexico, Oregon, Utah, and Washington, TIP Sheet for NHSN COVID-19 Enrollment: SAMS and Facility Enrollment. Accessed at https://comagine.org/sites/default/files/resources/TIP- Sheet-NHSN-Enrollment.pdf on November 17, 2022. 30Telligen and Medicare QIN-QIO, NHSN Access Level Guide. Accessed at https://www.telligenqiconnect.com/wp-content/uploads/2022/02/NHSN-Access-Level-Document_Current-Process.pdf on November 17, 2022. 31CDC, Secure Access Management Services (SAMS) Frequently Asked Questions, September 15, 2021. Accessed at https://auth.cdc.gov/sams/samsfaq.pdf?disp=true on November 17, 2022. 32CDC, New to NHSN? Guided Steps to Enroll Your Facility, p. 21. Accessed at https://www.cdc.gov/nhsn/pdfs/training/enroll/nhsn-facility-enrollment-training.pdf on July 5, 2023. 33Comagine Health and Medicare QIN-QIO for Idaho, Nevada, New Mexico, Oregon, Utah, and Washington, TIP Sheet for NHSN COVID-19 Enrollment: SAMS and Facility Enrollment. Accessed at https://comagine.org/sites/default/files/resources/TIP- Sheet-NHSN-Enrollment.pdf on November 17, 2022. 34Telligen and Medicare QIN-QIO, NHSN Access Level Guide. Accessed at https://www.telligenqiconnect.com/wp-content/uploads/2022/02/NHSN-Access-Level-Document_Current-Process.pdf on November 17, 2022. 35CDC, New to NHSN? Guided Steps to Enroll Your Facility, p. 21. Accessed at https://www.cdc.gov/nhsn/pdfs/training/enroll/nhsn-facility-enrollment-training.pdf on July 5, 2023. 36 CDC, About SAMS. Accessed at https://www.cdc.gov/nhsn/sams/about-sams.html on November 17, 2022. 37 CDC, About NHSN Help Desk. Accessed at https://www.cdc.gov/nhsn/about-nhsn/helpdesk.html on September 16, 2021. 38 CDC, About NHSN Help Desk. Accessed at https://www.cdc.gov/nhsn/about-nhsn/helpdesk.html on September 16, 2021. 39 Social Security Act §§ 1819, 1919. 4085 Fed. Reg. 27550 (May 8, 2020); CMS, Interim Final Rule Updating Requirements for Notification of Confirmed and Suspected COVID-19 Cases Among Residents and Staff in Nursing Homes, May 6, 2020, p. 3. Accessed at https://www.cms.gov/files/document/qso-20-29-nh.pdf on November 10, 2021. 41 85 Fed. Reg. 27627 (May 8, 2020); 42 CFR § 483.80. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Endnotes | 56 42CDC, NHSN E-Newsletter, March 2020, p. 13. Accessed at https://www.cdc.gov/nhsn/pdfs/newsletters/nhsn-nl-mar20-508.pdf on June 4, 2021. 43CDC, NHSN E-Newsletter, June 2020, p. 15. Accessed at https://www.cdc.gov/nhsn/pdfs/newsletters/nhsn-nl-jun20-508.pdf on June 4, 2021. 44 86 Fed. Reg. 26336 (May 13, 2021); 42 CFR § 483.80. 45 85 Fed. Reg. 54824 (September 2, 2020); 42 CFR § 488.447. 46OIG, CMS's COVID-19 Data Included Required Information From Most Nursing Homes, but CMS Could Take Actions To Improve Completeness and Accuracy of the Data (A-09-20-2005) September 1, 2021. 47 OIG analysis of internal documents provided by CDC, June 2023. 48CDC, COVID-19 Module Long Term Care Facility: Resident Impact and Facility Capacity Pathway, June 2023. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/57.144-res-blank-p.pdf on August 16, 2023. 49CDC, COVID-19 Module Long Term Care Facility: Staff and Personnel Impact, June 2023. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/57.145-staff-blank-p.pdf on August 16, 2023. 50CDC, Weekly COVID-19 Vaccination Cumulative Summary for Residents of Long-Term Care Facilities, June 2023. Accessed at https://www.cdc.gov/nhsn/forms/COVIDVax.LTC_.Resident.FORM_May2022-508.pdf on August 16, 2023. 51CDC, Healthcare Personnel COVID-19 Vaccination Cumulative Summary, June 2023. Accessed at https://www.cdc.gov/nhsn/forms/COVIDVax.HCP_.FORM_May2022-508.pdf on August 16, 2023. 52 42 CFR § 483.80. 53 CDC, National Healthcare Safety Network (NHSN) – Long-term Care Facility COVID-19 Module, May 2020, p. 22, 25-28. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/ltcf-covid19-module-508.pdf on June 22, 2021. 54 CDC, National Healthcare Safety Network (NHSN) – Long-term Care Facility COVID-19 Module, May 2020, op. cit., p. 21. 55CMS, Interim Final Rule Updating Requirements for Notification of Confirmed and Suspected COVID-19 Cases Among Residents and Staff in Nursing Homes, op. cit., p. 3. 56CMS, CMS/CDC Nursing Home COVID-19 Data Quality Assurance Process. Accessed at https://data.cms.gov/sites/default/files/2021-07/covid-19-nursing-home-data-quality-assurance-process_0.pdf on February 2, 2022. 57CMS, CMS/CDC Nursing Home COVID-19 Data Quality Assurance Process. Accessed at https://data.cms.gov/sites/default/files/2021-07/covid-19-nursing-home-data-quality-assurance-process_0.pdf on February 2, 2022. 58CMS published an interim final rule on November 5, 2021, that established COVID-19 vaccination requirements for staff at certain Medicare- and Medicaid-certified providers and suppliers. See 86 Fed. Reg. 61555 (November 5, 2021). 59CDC, Nursing Home COVID-19 Data Dashboard. Accessed at https://www.cdc.gov/nhsn/covid19/ltc-report-overview.html on September 15, 2021. 60CMS, COVID-19 Nursing Home Data. Accessed at https://data.cms.gov/covid-19/covid-19-nursing-home-data on January 3, 2022. 61CMS, COVID-19 Nursing Home Data. Accessed at https://data.cms.gov/covid-19/covid-19-nursing-home-data/data on January 27, 2023. 62CMS, CMS/CDC Nursing Home COVID-19 Data Quality Assurance Process. Accessed at https://data.cms.gov/sites/default/files/2021-07/covid-19-nursing-home-data-quality-assurance-process_0.pdf on February 2, 2022. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Endnotes | 57 63CMS, CMS Launches New Medicare.gov Tool to Compare Nursing Home Vaccination Rates, September 21, 2021. Accessed at https://www.cms.gov/newsroom/press-releases/cms-launches-new-medicaregov-tool-compare-nursing-home-vaccination- rates on January 25, 2023. 64OIG, CMS's COVID-19 Data Included Required Information From Most Nursing Homes, but CMS Could Take Actions To Improve Completeness and Accuracy of the Data (A-09-20-02005) September 1, 2021. 65OIG, COVID-19 Had a Devastating Impact on Medicare Beneficiaries in Nursing Homes During 2020 (OEI-02-20-00490) June 22, 2021. 66OIG, Onsite Surveys of Nursing Homes During the COVID-19 Pandemic: March 23-May 30, 2020 (OEI-01-20-00430) December 17, 2020. 67OIG, More Than a Thousand Nursing Homes Reached Infection Rates of 75 Percent or More in the First Year of the COVID-19 Pandemic; Better Protections Are Needed for Future Emergencies (OEI-02-20-00491) January 19, 2023. 68OIG, Workplan Summary: Meeting the Challenges Presented by COVID-19: Nursing Homes. Accessed at https://oig.hhs.gov/reports-and-publications/workplan/summary/wp-summary-0000474.asp on October 7, 2023. 69Minnesota Department of Health, Reporting Point of Care COVID-19 Test Results through the CDC National Healthcare Safety Network (NHSN), October 26, 2020. Accessed at https://www.leadingagemn.org/assets/docs/10_26_20_NHSN_POC_Test_Reporting_LTC.pdf on February 2, 2023. 70CDC, COVID-19 Pandemic Response, Laboratory Data Reporting: CARES Act Section 18115, March 8, 2022, p. 5. Accessed at https://www.cdc.gov/coronavirus/2019-ncov/downloads/lab/HHS-Laboratory-Reporting-Guidance-508.pdf on February 2, 2023. 71A Quality Improvement Organization is a group of health quality experts, clinicians, and consumers organized to improve quality of care for people with Medicare coverage. See CMS, Quality Improvement Organizations. Accessed at https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityImprovementOrgs on December 7, 2022. 72 CDC, July 2020 LTCF Newsletter, July 6, 2021. Accessed at https://www.cdc.gov/nhsn/ltc/newsletters/jul20-newsletter- ltc.html on November 17, 2022. 73CDC, "LTCF COVID-19 Module: Resident Impact and Facility Capacity Surveillance Reporting Pathway," YouTube, April 2, 2021. Accessed at https://www.youtube.com/watch?v=mrGOM4-hgAo on December 13, 2022. 74CDC, December 2020 LTCF Newsletter, December 21, 2021. Accessed at https://www.cdc.gov/nhsn/ltc/newsletters/dec20- newsletter-ltc.html on December 13, 2022. 75CDC, "LTCF COVID-19 Module: Therapeutics Surveillance Reporting Pathway," YouTube, April 19, 2021. Accessed at https://www.youtube.com/watch?v=ZTzRk7-3Qak on December 15, 2022. 76CDC, LTCF COVID-19 Module: Updates, July 2021. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/rel955-update- 508.pdf on December 15, 2022. 77CDC, LTCF COVID-19 Module: Surveillance Pathway Updates, May 2022. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/covid19-mod-surv-path-508.pdf on December 15, 2022. 78 CDC, LTCF COVID-19 Module: Surveillance Pathway Updates, June 2023. Accessed at https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/NHSN-COVID-19-11.4-updates-for-May-30th-recording-final.pdf on June 13, 2023. 79CDC, 5-Step Enrollment for Long-term Care Facilities. Accessed at https://www.cdc.gov/nhsn/ltc/enroll.html on February 7, 2023. CDC, User Guide for CDC's SAMS Partner Portal. Accessed at https://auth.cdc.gov/sams/SAMSUserGuide.pdf?disp=true on 80 November 17, 2022. 81CDC, Overview for CDC's SAMS Identity Verification Process. Accessed at https://auth.cdc.gov/sams/IdentityVerificationOverview.pdf?disp=true on November 17, 2022. 82CDC, User Guide for CDC's SAMS Partner Portal. Accessed at https://auth.cdc.gov/sams/SAMSUserGuide.pdf?disp=true on November 17, 2022. CDC Has Improved the Nursing Home Reporting Process for COVID-19 Data in NHSN, but Challenges Remain OEI-06-22-00030 Endnotes | 58