This dates me, but artificial intelligence (AI) reminds me of the microwave. First, I learned what a microwave was. Next, I discovered the cooking problems it could solve, such as defrosting frozen foods in minutes, not hours. Finally, I learned to operate it. The same process happened with AI. What is AI? What can it do for Freedom of Information Act (FOIA) offices at government agencies? How does it work? There are uncertainties too, like there were with the microwave. Would it irradiate someone while warming their soup? FOIA professionals are nervous about bias and hallucinations with AI, but they’re also nervous about the record 928,353 FOIA requests received by U.S. federal agencies in 2022.

Professor Jason R. Baron of the University of Maryland College of Information Studies is convinced you must spell “FOIA” with “AI.” “Without AI methods being deployed, well-meaning FOIA offices throughout large agencies will simply be overwhelmed with the task of complying with FOIA deadlines and meeting the requirements for reasonable searches being conducted,” he writes.

This NewsBreak describes how the U.S. Department of State and other agencies are spelling FOIA with AI.

THE STATE DEPARTMENT’S PROJECT

At the Sept. 7, 2023, meeting of the National Archives and Records Administration’s (NARA) FOIA Advisory Committee, Eric Stein, deputy assistant secretary of the Office of Global Information Services in the State Department, presented the department’s project, Piloting Machine Learning for Freedom of Information Act (FOIA) Requests. The project consisted of three small pilots that used AI in three FOIA tasks: taking records requests, finding records, and confirming that records are releasable under the law.

The FOIA Customer Experience (CX)—FOIA Assistant pilot, which wraps up this month, considered how AI could improve a citizen’s experience on the State Department’s FOIA website. The State Department used AI to analyze data from the website. Requesters were greeted with an AI pop-up with referrals to already-posted records or pending FOIA requests for similar records. Requesters could choose a notification when records were released in response to the pending requests. When the AI pop-up couldn’t help requesters, it asked more questions. The State Department plans to share the results of this pilot later this month.

Another U.S. federal agency, the Office of Community Oriented Policing Services, deployed an AI chatbot to answer questions or direct citizens to records on its website, according to the “Department of Justice Chief FOIA Officer Report 2022.”

EXCITING NEW METHODS FOR SEARCHING WITH AI

The State Department’s Grouping Similar FOIA Cases and Parts of Cases—One Search for Many Cases pilot also ends this month. At the FOIA Advisory Committee meeting, Stein explained the pilot’s focus on searching: The State Department knew that FOIA requesters filed many almost identical records requests on events of significance. The State Department used AI to group similar requests, articulate a machine-readable query, and conduct a single search of its records, rather than running many separate searches.

After approval by FOIA staff, incoming records requests were translated into machine-readable queries by natural language processing. The queries were fed into a centralized records archive that identified records potentially relevant to FOIA requests. AI determined which records were already publicly available, thus needing no further scrutiny. This pilot also tested the identification of classified and sensitive records within search results. “We’re not sure we’re going to succeed, but we’re going to try,” said Stein. The State Department will share results in March.

NARA’s Freedom of Information Act (FOIA) Discovery AI Pilot—as described in the Inventory of NARA Artificial Intelligence (AI) Use Cases (download required)—used AI-based vector and content similarity searching. AI, trained with records already released to the public, detected content similarities between new records requests and stored records. As of September 2023, NARA had completed initial development and was testing the technology.

CAN WE RELEASE THIS RECORD? A SUCCESSFUL PILOT OF RECORDS DECLASSIFICATION

For FOIA professionals, exemptions, which designate specific records as confidential, are tedious and time-consuming. FOIA pros must identify and protect these records from public release. There are nine exemptions for federal agencies. FOIA Exemption 1 restricts access to records that are classified to protect national security. Could AI automate deciding if 100,000 classified State Department cables were eligible for declassification? If so, they could be made available to the public. The State Department’s Machine Learning Declassification Review Pilot investigated whether discriminative AI could do the job.

“Discriminative AI is a model that learns and distinguishes similarities and differences in data to predict classifications,” said Giorleny Altamirano Rayo, chief data scientist at the State Department. Her team trained machine-learning models with cables already declassified by human experts. The models gave each classified cable a confidence score from 0 to 1. All cables with a score of 0 to 0.1 could be declassified and released to the public without further review. Cables scored from 0.9 to 1 stayed classified. Cables between 0.1 and 0.9 were sent to human experts for final decisions. These human judgments trained the machine-learning models for the next round of declassifications. Rayo said, “This small-scale pilot offers us a proof of concept to integrate this technology into a routine declassification process. … We will use this process in future years.”

Government agencies also learn when AI doesn’t fit for the detection of exempt records. The Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics evaluated AI for finding confidential personal data in records. In August 2023, a CDC spokesperson said, “Ultimately, this software did not meet the needs of the office in an efficient and timely manner and it was determined the tool was not the right fit (for the) intended use.”

ARTIFICIAL INTELLIGENCE STILL NEEDS HUMAN INTELLIGENCE

Adam Marshall, senior staff attorney at the Reporters Committee for Freedom of the Press, speaking at the National FOI Summit ’23, appealed for human guardrails when AI is used for FOIA because AI has a propensity for inheriting biases from its training data. Marshall pushed for transparency by government agencies and said, “Whenever AI is involved, that should be disclosed to the requester.” The State Department’s Stein agreed that human FOIA professionals must root out any bias and other legal violations that may be present in AI-generated FOIA responses and said each FOIA office must have a published policy about its deployments of AI.

LEARNING ABOUT AI FOR FOIA AND NEXGEN 2.0

At the National FOI Summit ’23, Michael Sarich, FOIA director of the Department of Veteran Affairs, encouraged FOIA professionals to read everything AI and discuss AI with colleagues and AI experts. Federal FOIA staff can learn more by following NARA’s Freedom of Information Act (FOIA) Advisory Committee and the Chief FOIA Officers Council Technology Committee. Also, send requests for information to companies with intriguing AI technology, said Baron.

In 2022, the Chief FOIA Officers Council, NARA’s Office of Government Information Services, and the Department of Justice’s Office of Information Policy convened 22 companies, some with AI technology, for a virtual NexGen FOIA Tech Showcase. The Chief FOIA Officers Council is planning NexGen 2.0 for this spring, where one can learn about the companies and their newest FOIA technologies, including AI, all in one place.

Just like the microwave became indispensable in the kitchen, AI could become indispensable for FOIA work. For this to happen, future pilot projects can investigate some formidable FOIA tasks. Searching through email archives is one. Another is removal of confidential information from within records. There is already good redaction software for finding and removing the xxx-xx-xxxx pattern of Social Security numbers. A future project could test AI for removal of unstructured confidential information, such as a back-and-forth discussion of a personnel matter. And is there a role for generative AI in FOIA work?

Let FOIA technology expert Baron have the last word: “Making progress in solving the problem of access through the use of AI methods is a key to ensuring that FOIA continues to provide meaningful access to government records and the history of our Nation.”

LESSONS LEARNED FROM THE STATE DEPARTMENT’S PRESENTATION

The State Department’s “Piloting Machine Learning for Freedom of Information Act (FOIA) Requests” presentation from the Sept. 7, 2023, FOIA Advisory Committee Meeting offers the following lessons learned:

• Managing data and records well is critical to success.
• Understand available data; develop standards now for future use.
• Learn about AI. 
  • Recognize what you do and do not know.
  • Do research or reach out to colleagues. …
• Scope—Start small, explore one question in pilot(s).
• Identify manual processes that could be automated.
• Take (controlled) risks—don’t be afraid to fail.
• Consider quality controls such as human reviews before you start.
• Consider bias, privacy, sensitivity of information, [and] legal requirements at the start of your project.
• [Have d]iscussions about integration of various IT tools using AI and machine learning capabilities.
• Resources are required for pilots and to implement changes.
• Be flexible and open to a variety of results.
• Be open to sharing successes and failures.
• Results in one pilot or project may not be applicable to other programs ... but they may.