WHAT’S BEHIND THE AI HYPE

In the session Cutting Through the Hype: AI and Public Records, we found out if government agencies really are using AI technology to respond to public requests for records and information.

Dillon Bergin, data reporter for MuckRock Foundation, investigated federal agencies’ use of AI. Bergin asked MuckRock’s users to volunteer to parse 120 Chief FOIA Officer Reports from 2023 and 2024. Chief FOIA Officers for all federal agencies must submit these reports, detailing improvements to their FOI processes, to the Department of Justice. The MuckRock volunteers zeroed in on their answers to the question, “Does your agency currently use any technology to automate record processing?”

In spite of the AI hype, Bergin’s team found few federal agencies testing AI for public records requests. Agencies use machine learning and automation for steps such as the culling of duplicates and the detection of confidential information, but not “what we think of as artificial intelligence in the age of large language models and generative AI,” said Bergin. Even with constricted budgets, however, Chief FOIA Officers were interested in AI and in conversations with AI technology companies. For more on this research, read Bergin’s article.

At the state government level, we learned about reality rather than hype from Amy Glasscock, program director of innovation and emerging issues at the National Association of State Chief Information Officers (NASCIO), who explained the findings of NASCIO’s survey of state governments’ use of generative AI. States’ most common actions to date were the formation of AI advisory committees, the publication of policies and procedures, the development of ethical guardrails, and, in more states than not, the inventorying of actual generative AI projects. So far, these projects support internal functions, not citizen interactions such as records requests.

NASCIO’s survey found that over the next 5 years, states will begin to use generative AI for citizen interactions. Glasscock explained generative AI’s potential for improving records requests. AI-powered virtual assistants and auto-generated responses could improve efficiency and speed of responses. Using generative AI, state agencies could retrieve information faster and be prepared to scale up for surges in requests, such as after newsworthy events. Generative AI could also overcome obstacles to citizens’ access, such as language barriers and disabilities.

NASCIO identified poor data quality as a major challenge to deployments and a guarantee of wrong responses to records requests if answered by generative AI. State governments also suffer from a “worker crisis,” said Glasscock. States’ IT professionals lack the skills to work with generative AI and strain to keep up with new privacy and AI regulations from the pens of their own state legislatures. Over the next half decade, NASCIO’s research foresees state IT workers either retiring or reskilling with new generative AI prowess and legislatures lifting restrictions on the use of generative AI.

It’s tricky. States want to be “progressive and innovative without making a lot of mistakes,” said Glasscock. For more information, read “Generating Opportunity: The Risks and Rewards of Generative AI in State Government.”

NOT HYPE: USING AI TO REQUEST PUBLIC INFORMATION

On the requester side, MuckRock’s Bergin said, “I’m really excited about the ability of large language models to reference information that you give them. … This is RAG—retrieval-augmented generation.” According to Wikipedia, RAG “is a technique that grants generative artificial intelligence models information retrieval capabilities. It modifies interactions with a large language model (LLM) so that the model responds to user queries with reference to a specified set of documents, using this information to augment information drawn from its own vast, static training data.”

Bergin cited occasions when a journalist hard at work on a story or a citizen prepping for a public meeting doesn’t know which public records they need or where to ask for them. With RAG, generative AI could draw on LLMs augmented with encyclopedic information about public records. Journalists and citizens could use the RAG tools to write records requests.

In the session Demos: AI-Based Tools, Charles Minshew, digital storytelling editor for The Atlanta Journal-Constitution, prompted ChatGPT to write a request for ridership data for Atlanta city buses. Like magic, ChatGPT wrote a request to the correct agency, the Metropolitan Atlanta Rapid Transit Authority, and cited the correct law, Georgia’s public records statute. “I can copy and paste and use it very quickly,” said Minshew. Minshew next asked ChatGPT to write the same request, but this time for Houston, Texas. ChatGPT rewrote the request before our eyes.

Minshew then demonstrated the power of RAG with his own Georgia Records Guide, a custom GPT he is testing at The Atlanta Journal-Constitution. Minshew fed ChatGPT a specified set of documents, in this case, the public records guides of the Georgia First Amendment Foundation. When queried, the Georgia Records Guide searches the guides’ text plus preselected internet sites, generates a summary of relevant public records, and writes the request. Minshew also created Record Retention Advisor, a custom GPT fed Georgia’s records retention schedules. The Record Retention Advisor tells how long Georgia agencies must keep records. “I would love to do this for all 50 states,” said Minshew.

Scott Horn, co-director of the Kentucky Open Government Coalition, demonstrated his Kentucky Open Records Assistant, a RAG implementation built on ChatGPT for answering questions about Kentucky Attorney General decisions in public records cases. Horn took raw text of decisions dating back to 1977 and converted this unstructured text into neat rows and columns in a spreadsheet with Python code written by ChatGPT and other tools. ChatGPT’s generative AI also traces citations to these decisions and generates brief summaries. Like Minshew, Horn was enthusiastic about the promise of custom GPTs where he can provide his own data sources for teaching ChatGPT. “The tools are really coming along,” he said.

NEWS FROM THE NFOIC

In addition to AI developments, we heard more news at 2024 National FOI Summit. The NFOIC announced four inductees into the State Open Government Hall of Fame and shared eight winning papers in its NFOIC FOI Research Competition.

Secrecy Tracker, a new, free website for tracking public records and open meetings legislation in all 50 states, was unveiled. Developed by the Joseph L. Brechner Freedom of Information Project, Secrecy Tracker, now in beta, is scheduled for official debut in January.

In a session on the latest FOI legislation, Horn noted a Kentucky Supreme Court decision that expands public access to police records in Kentucky. Liz Wagenseller, executive director of the Pennsylvania Office of Open Records, expressed concern “that generative AI is going to the change the game” and open the door to vexatious requesters who want to submit a large number of requests at one time. The Pennsylvania Office of Open Records, while encouraging agencies to make a good faith effort to answer, suggests they ask high-volume requesters for an extension of the due dates in Pennsylvania’s Right-to-Know Law.

Agencies also can make the automation of requests difficult with reCAPTCHA tests and formal protocols and forms for records requests. More broadly, the Pennsylvania legislature may have to reconsider the definition of a legitimate requester. Still, said Wagenseller, her office wants “to make sure with the way legislation is drafted that the true intent of Pennsylvania’s Right-to-Know Law is kept intact.”

What if federal and state government agencies use generative AI to create new kinds of public records? Richard Varn, executive director of the Coalition for Sensible Public Records Access, said, “If AI is making a decision, what now does the public records law have to say and do for [the public] to understand what the heck is going on in government, how that got decided, and was that the right decision?” Varn predicted that this uncertainty could cause legislatures to reform FOI laws to cover records created by generative AI.

KEEPING THE HUMAN-IN-THE-LOOP

According to Wikipedia, human-in-the-loop is a model that requires human interaction. “It’s interesting to watch this evolution in the AI world to see if we are really trying to make the human-in-the-loop more powerful, which is a model I really like,” said Varn. “Is AI being used to make you a super-reporter, a super-records advocate, or a super-government manager?” NASCIO’s Glasscock agreed on the importance of the human-in-the-loop. About the possibility of generative AI responding to public records requests with wrong information, she said that with human-in-the-loop, “that shouldn’t be happening.”

With humans in the loop, generative AI is a tool for open government and freedom of information, according to the analysis of open government advocates at the 2024 National FOI Summit. At Sunshine Fest 2025 in March, advocates will assemble in person in Washington, D.C., and write an action plan to help the public understand the importance of their right to public information. They will, said Richard T. Griffiths, a retired journalist and lecturer on journalism ethics and investigative journalism, “beat the drum loudly.”