Assessing Freedom of Information Act Compliance through the National Archives and Records Administration’s 2022 Records Management Self-Assessment

Published August 7, 2023 | Download the assessment

EXECUTIVE SUMMARY

The Office of Government Information Services (OGIS) began collaborating with the Chief Records Officer (CRO) for the U.S. Government at the National Archives and Records Administration (NARA) in 2016 to gather information from across the government about Freedom of Information Act (FOIA) administration. In the CRO’s 2022 Records Management Self-Assessment (RMSA), OGIS included questions to measure obstacles, successes, and pace of innovation in FOIA administration. This data complements the observations OGIS makes as the FOIA Ombudsman, working to improve the FOIA process for all.

The 2022 RMSA—administered to agency records officers from January 9, 2023, to March 10, 2023—is an annual agency self-assessment and reporting tool developed and managed by the CRO. The RMSA included six questions regarding FOIA administration. One question sought to examine the collaboration between Chief FOIA Officers and Agency Records Officers and was repeated from the 2021 RMSA to track trends. One question pertained to the COVID-19 pandemic and its continuing impacts in 2022 on FOIA processing; another sought to quantify the frequency with which agencies proactively disclose records; and the other three questions pertained to e-discovery, seeking to assess the current situation, identify obstacles to implementation, and establish a baseline against which to compare data from future RMSAs.

Key results include[1]:

• Across the government, 79 percent of agencies reported either minimal or no pandemic-related backlog, while 18 percent of agencies reported a moderate or significant pandemic-related backlog.[2]
• Of the respondents, 73 percent reported using e-discovery tools in their FOIA searches, while 19 percent reported not using such tools. The most common use of e-discovery in FOIA searches involves requests for email records—88 percent of those agencies that use e-discovery use it for this reason. The most common reason why e-discovery is not used for FOIA is that it is unavailable at certain agencies—63 percent of FOIA offices that do not use e-discovery cite this as the reason.
• A majority of respondents (57 percent) make proactive disclosures “as needed” rather than routinely. A quarter (25 percent) of respondents reported making proactive disclosures on a routine basis, either annually (13 percent), quarterly (7 percent) or monthly (5 percent).

BACKGROUND

OGIS began a collaboration with the CRO beginning with the 2016 RMSA to assess government-wide issues related to FOIA compliance. The CRO has issued the RMSA since 2009 to determine the extent to which agencies are complying with statutory and regulatory records management requirements, and the issues agencies face in compliance. It was a natural fit for OGIS to include FOIA questions in the RMSA data collection because a strong records management program—which allows agencies to find records responsive to a FOIA request—is essential to a successful FOIA program. In working with the CRO, OGIS has leveraged investments by NARA in survey technology and the CRO’s expertise in collecting self-reported compliance information.

The RMSA’s high response rate in the last few years has helped OGIS better understand FOIA administration across the government and complements the observations we make through our other activities, including providing dispute resolution services to requesters and agencies, assessing FOIA compliance, and leading the federal FOIA Advisory Committee and the Chief FOIA Officers Council. Since Calendar Year (CY) 2018, the RMSA response rates have consistently been at or above 95 percent, falling below—to 92 percent—in CY 2020, when the self-assessment coincided with the COVID-19 pandemic. This year’s response rate was 95 percent.

The 2022 RMSA included six questions relating to FOIA. One question, which was repeated from the 2021 RMSA, sought to assess how Agency Records Officers work with Chief FOIA Officers across the government. Another question asked directly about the COVID-19 pandemic’s continued impact on FOIA operations. Three of the remaining questions assessed use of e-discovery in FOIA searches, and the final question asked about frequency of proactive disclosure under FOIA.

The 2022 RMSA FOIA questions received responses from 261 respondents, including from all 15 Cabinet-level departments, departmental components and independent agencies. Not all 261 respondents answered all the FOIA questions because some agencies chose to skip certain questions while respondents in the judicial branch are subject to the Federal Records Act but not to the FOIA. A complete list of the six 2022 RMSA questions, each of which is discussed in detail below, is available below in the Methodology Section.

DISCUSSION OF RMSA RESULTS

A Rebound from COVID-19

Since March 2020, when the Office of Personnel Management (OPM) ordered maximum telework flexibilities across the federal government in response to the COVID-19 pandemic, agency FOIA programs have largely adapted to the move to a full-time work-from-home environment.  The move affected FOIA processing governmentwide, particularly at agencies that were not telework-ready, agencies that work with classified material, and agencies with largely paper records. The results from the 2022 RMSA show that a majority of agencies (79 percent) have rebounded from the pandemic—reporting minimal or no pandemic-related negative impact to the backlog. However, a sizable minority (18 percent) report a continuing moderate or significant negative impact to the backlog caused by the pandemic.

Q56. How much impact does the ongoing COVID-19 pandemic still have on your agency’s FOIA processing?

Of the 261 respondents who answered this question—53 percent answered “None - completely meets or exceeds pre-pandemic levels of backlogs,” and 26 percent answered “Minimal negative impact to backlog continues.” Thirteen percent answered “Moderate negative impact to backlog continues” and 5 percent answered that a “significant negative impact to backlog continues.” Another 3 percent were agencies which are not subject to FOIA.

Enlarge

Figure 1

2022 RMSA Responses to Question 56 “How much impact does the ongoing COVID-19 pandemic still have on your agency’s FOIA processing?”

As shown in Table 1, the comments of the 2022 RMSA respondents who volunteered explanations centered on two themes: the pandemic exacerbating resource challenges, and an influx of FOIA requests seeking records about the government’s pandemic response.

Table 1: Selected Data from 2022 RMSA voluntary explanations Responses to Question 56 “How much impact does the ongoing COVID-19 pandemic still have on your agency’s FOIA processing?”

Observations

OGIS has observed that several agencies have seen an increase in FOIA requests asking for records that may shed light on the federal government’s response to the pandemic. This surge of requests indicates why some agencies continue to experience a significant or moderate impact to their FOIA backlog despite no longer facing pandemic-related obstacles to processing requests.  

Agency Records Officers and Chief FOIA Officers

Q57. Which of the following describes the working relationship between the Agency Records Officer and the Chief FOIA Officer?

The second FOIA question in the 2022 RMSA asked respondents to describe the working relationship between the Agency Records Officer (ARO) and Chief FOIA Officer (CFO) at their agency by choosing all of the statements that applied.

OGIS has long been interested in the relationship between AROs, senior officials within each agency whose responsibilities include oversight of records management, and CFOs, senior officials within each agency whose responsibilities include “efficient and appropriate compliance” with FOIA and recommending FOIA improvements to the head of the agency.

The question about the relationship between the two is repeated from the 2020 and 2021 RMSAs. The 2019 RMSA asked if the ARO and staff received FOIA training, and the 2017 RMSA asked how closely the ARO and the CFO worked together.

Between 2021 and 2022, answers to three questions dropped by 5 percentage points or more.

In the 2022 RMSA:

• Forty-four percent of respondents reported that the ARO and CFO work together to identify programs or offices most likely to have responsive records, a 6-percentage point decrease over 2021’s response of 50 percent.
• Forty-three percent of respondents report that the ARO and CFO provide training on records management and FOIA to each other’s staff, a 5-percentage point drop from 2021’s result of 48 percent.
• Forty-two percent of respondents reported that training programs include the importance and relationship between FOIA and records management, which is 7 percentage points lower than last year’s 49 percent.

Other findings in the 2022 RMSA include: 54 percent reported that the ARO and CFO work together on Information Technology (IT) requirements that benefit both programs; 44 percent reported that the two work together on high-profile or complex FOIA requests; 38 percent said that the ARO and CFO coordinate search terms to identify responsive records; 13 percent responded “Other, please explain;” and 5 percent answered, “None of the above.” Two final options covered two reasons why the relationship between the ARO and CFO was not applicable: Agency Records Officer and the Chief FOIA Officer are the same person (5 percent), and the Agency is exempt from FOIA and thus does not have a Chief FOIA Officer (3 percent).

Observations

OGIS has long recognized the importance of strong records management as a solid foundation for an efficient and compliant FOIA program and smoother FOIA process.

OGIS notes with concern the 7-percentage point drop in respondents who reported that training programs include the importance and relationship between FOIA and records management and the 5-percentage point drop in respondents who reported that the ARO and CFO provide training on records management and FOIA to each other’s staff. Also concerning is the 6-percentage point decrease in collaborating to identify offices most likely to have responsive records.

What is unclear is whether previous training on FOIA and records management as well as previous identification of offices most likely to have responsive records affected FOIA processes at these agencies. While the placement of FOIA and records management programs within agency organizational structures varies, records management and FOIA staff should work together to train agency staff in both disciplines, and to procure and deploy IT systems that help agencies carry out their missions and comply with both records management and FOIA requirements.

Enlarge

Figure 2

2022 RMSA Responses to Question 57 "Which of the following describes the working relationship between the Agency Records Officer and the Chief FOIA Officer?"

Use of E-discovery Tools

In 2020, the FOIA Advisory Committee recommended that the Department of Justice, Office of Information Policy (OIP), provide guidance to agencies on the use of e-discovery tools to assist agencies in their searches of electronic records in response to FOIA requests. (Recommendation 2020-11). Agencies continue to report in their CFO Reports on their efforts to leverage e-discovery and other tools, while OIP continues to encourage agencies to leverage technology for greater efficiency. This issue is of great interest to the FOIA community, and the Chief FOIA Officers Council’s Technology Committee continues to study ways e-discovery can improve FOIA processing. The 2022 RMSA included questions about e-discovery to assess the current landscape across FOIA programs governmentwide.

The results from the 2022 RMSA demonstrate the work agencies have put into e-discovery, and the work left to do. Three questions asked agencies about their use of e-discovery tools or reasons for lack of use. This data suggests areas where there may be room for improvement in leveraging such technology across federal agencies.

Q58. Does your agency use e-discovery tools to search for records when responding to FOIA and/or Legal Discovery?

The question asking whether agencies use e-discovery tools for FOIA and/or Legal Discovery received a total of 261 responses. Of the respondents, 73 percent answered yes, and 19 percent answered no, with an explanation. Five percent answered that they did not know.

Since 2020, there has been minimal change in the percentage of respondents who use e-discovery tools to search for records responsive to a FOIA request. On the 2020 RMSA, when this question was last asked, 72 percent answered "yes." In 2022, there was a 1-percentage point increase to 73 percent of respondents answering “yes.” The majority of FOIA programs that do not use e-discovery tools report facing  obstacles at the agency level, either because the tools are not available agency-wide or there are a lack of resources to deploy the tools to the FOIA program. (See discussion, below, regarding Question 60.) These results come with significant improvement in technology over the past few years, and great interest in Artificial Intelligence as a tool for searching, sorting, and redacting.

Enlarge

Figure 3

2022 RMSA Responses to Question 58 "Does your agency use e-discovery tools to search for records when responding to FOIA and/or Legal Discovery?"

Several comments offered by respondents who "“No, please explain” stated that they did not know what e-discovery is. These responses show that there is an opportunity to educate FOIA professionals on the potential benefits of e-discovery so they are aware of tools that may aid their FOIA programs.

Q59. If Yes: For what purposes are e-discovery tools used? (Choose all that apply)

The question asking agencies that use e-discovery how they use it received 190 responses. Eighty-eight percent said they use e-discovery to search for records involving FOIA requests for email records; 78 percent for lawsuit-related requests; 73 percent for legal discovery or third-party subpoena requests; 70 percent for de-duplication of records in responding to requests; 67 percent for managing legal holds; 63 percent for FOIA responses to requests for records other than email records; 44 percent for internal research for or by staff; and 23 percent for knowledge management. Since this question only applied to respondents who answered yes on Q58, there were no respondents for whom this was not applicable, nor did any select “none of the above.”

Between 2020 and 2022, there was a trend for FOIA programs that have access to e-discovery to use it to search for more categories of records.

Enlarge

Figure 4

Comparison of 2022 RMSA Responses to Question 59 and 2020 RMSA Responses to the Same Question "For what purposes are e-Discovery tools used? (Choose all that apply)?"

Q60. Why are e-discovery tools not used to search for records? (Choose all that apply)

The question asking for reasons why e-discovery is not used to search for records got a total of 49 responses. The most common reason, selected by 63 percent of respondents, was that the discovery tools are not available at their agency. Thirty-seven percent responded “other” (with an explanation), 22 percent responded “cost,” 4 percent responded “limited number of licenses available,” and 6 percent did not know. 

In comparing responses to this question from the 2020 RMSA and the 2022 RMSA, the data  appears to indicate that many respondents who do not use e-discovery tools have gained an awareness of those tools. The most dramatic change between the two RMSAs was that while 56 percent of respondents selected ‘other, please explain’ on the 2020 RMSA, that response dropped to 37 percent on the 2022 RMSA, with a corresponding drop in the number of explanations indicating unfamiliarity with e-discovery. Additionally, the percentage of respondents who answered that “e-discovery tools are not available at my agency” rose from 51 percent to 63 percent over the two years, indicating a growing awareness of e-discovery tools and the obstacles to using them for FOIA searches.

As shown in Table 2, the “Other, please be specific” comments in the most recent RMSA show that many agencies that do not currently use e-discovery tools either do not need them because they receive so few FOIA requests so as to make e-discovery tools unnecessary or they are working to establish this technology in their agency.

Table 2: Selected Data from 2022 RMSA “Other, please explain,” Responses to Question 60 “Why are e-discovery tools not used to search for records. (Choose all that apply)”

Observations

The data indicate that e-discovery tools are widely used across government agencies to conduct certain FOIA searches. While there is significant room for improvement, the main take-away appears to be that FOIA programs will benefit as many hold-out agencies make e-discovery tools available to their FOIA professionals.

Proactive Disclosures

Q61. How often are you proactively (i.e., before receiving any FOIA requests for the information) making records public by posting information appropriate for the public?

The final FOIA question in the 2022 RMSA asked respondents to describe how often they proactively (i.e., before receiving any FOIA requests for the information) make records public by posting information appropriate for the public. There were 260 responses to this question. The most common response was “as needed” (57 percent); 13 percent of respondents reported making proactive disclosures annually; 5 percent reported making proactive disclosures monthly; and 7 percent reported making proactive disclosures quarterly.  The remaining options were: “other, please explain” (12 percent); “do not know” (3 percent) and “not applicable, my agency is exempt from FOIA” (3 percent).

As shown in Table 3, the “Other, please be specific” comments in the most recent RMSA illustrate the wide variety of agency assumptions regarding proactive disclosure. Agencies whose mission involves public outreach generally make data routinely available. Other comments support what OGIS has observed: that agencies often have limited resources to keep up with FOIA requests and have limited capacity to identify potential records for proactive release. The comments from those agencies indicate that they would follow the “rule of three” —the requirement under FOIA that an agency publicly post a record that has been requested three or more times[3]— if it were applicable or that they have embraced a “release-to-one -release-to-all” policy.

Table 3: Selected Data from 2022 RMSA “Other, please explain,” Responses to Question 61 “How often are you proactively making records public by posting information appropriate for the public?”

Enlarge

Figure 5

2022 RMSA Responses to Question 61 "How often are you proactively making records public by posting information appropriate for the public?"

Observations

OGIS has observed that a number of agencies struggle to respond to a high volume of FOIA requests, leaving little time or other resources for robust and routine proactive disclosures. As OGIS reported in February 2020, federal agencies generally comply with FOIA’s mandate to create procedures for posting documents on FOIA reading rooms, with more than 83 percent of respondents saying that their agencies have such procedures. More than half of the time, however, the responsibility for preparing the documents for posting rested with the FOIA staff.[5]

CONCLUSION

OGIS’s collaboration with the CRO in its annual RMSA helps us fulfill our statutory mission to “identify procedures and methods for improving compliance” under FOIA. For the 2022 RMSA, we are encouraged that the majority (79 percent) of respondent agency FOIA programs reported having minimal or no pandemic-related impact to the FOIA backlog.

We are hopeful that the continued implementation of e-discovery tools as agencies fully transition to electronic records management systems will bring about faster, less resource-intensive, and more comprehensive FOIA searches. This technology can help alleviate backlogs and improve FOIA administration throughout the federal government.

Additionally, the responses demonstrate that there is room for improvement in proactive disclosures under FOIA.

The 2022 RMSA has yielded data which informs the efforts of OGIS, the Chief FOIA Officers Council, and the FOIA Advisory Committee in working toward a FOIA process that works for all.

METHODOLOGY

The CRO’s office opened the 2022 RMSA on January 9, 2023, with a response deadline of March 10, 2023. The CRO’s office conducts the RMSA via an online survey tool that creates a unique link used to submit responses. Ninety-five percent of agencies that received the 2022 RMSA link completed the assessment in accordance with NARA’s responsibility to report on the state of federal records management. Each year, federal agencies are required to conduct an RMSA and submit their findings to NARA. The goal of the self-assessments is to determine whether federal agencies are compliant with statutory and regulatory records management requirements. A total of 261 agencies responded to the RMSA, but not all respondents answered the FOIA questions. The respondents included records officers at all Cabinet-level departments, departmental components, and independent agencies. Eight non-executive branch agencies that are not subject to FOIA took part in the assessment. CRO instructed agency Records Managers to consult with their agency FOIA Officers to answer the FOIA-related questions. (Percentages are rounded up and may not equal 100 percent.)

The full 2022 RMSA report is one of three reports required by the CRO, the results of which are included in the Federal Agency Records Management Annual Report to Congress, published on NARA’s website on the CRO’s RMSA web page. Questions 56-61 pertained to FOIA.

List of Statistical Results from 2022 RMSA FOIA questions

[1] Percentages may not equal 100 percent due to rounding and/or 'chose all that apply' questions.

[2] The other 3 percent of respondents are not subject to FOIA.

[3] “Each agency, in accordance with published rules, shall make available  for public inspection in an electronic format copies of all records, regardless of form or format that have been requested 3 or more times.” 5 U.S.C. § 552 (a)(2)(D)(ii)(II).

[4] FOIAonline, a system for tracking requests and appeals and providing a repository for released records, will be decommissioned by the Environmental Protection Agency on September 30, 2023.

[5] Office of Government Information Services, OGIS Report: Assessing FOIA Compliance through the 2018 National Archives and Records Administration’s Records Management Self-Assessment, February 11, 2020, https://www.archives.gov/files/ogis/assets/2018-foia-compliance-self-assessment-report.pdf.