For state and territorial archives that preserve or want to preserve emails, the Council of State Archivists (CoSA) comes to the rescue with “Be Prepared: Managing and Preserving Email in State and Territorial Governments” (download required), published in March 2023. This report, packed with useful information, emphasizes two guiding principles for the preservation of state and territorial emails with historical value—centralized governance and “no signal loss” (i.e., containing all email content, including subject lines, text, and attachments)—and describes the foundation for an email preservation program.

THE PREPARE PROJECT

“Be Prepared” is the latest product of the CoSA PREPARE: Preparing Archives for Records in Email project. CoSA is the association of government archives of all 50 states, five territories, and the District of Columbia. It is home to the State Electronic Records Initiative (SERI) for research on the archiving of digital records. The goal of CoSA PREPARE, which is within SERI and is funded by the Mellon Foundation through the University of Illinois’ Email Archives: Building Capacity and Community program, is “to develop and deliver a variety of capacity-building activities for email preservation and access,” its website notes.

“Be Prepared” draws from findings of the CoSA PREPARE project’s needs assessment survey of state and territorial archives’ email preservation capabilities, a focus group that tested mostly open source email preservation tools and released the “Phase 2 Testing Report” (download required), and CoSA’s ongoing direct assistance to archives. The needs assessment survey report summary says, “State and territorial archives need to build additional capacity to successfully deal with the massive influx of email records and other electronic messaging records expected in the near future.” According to the needs assessment survey report (download required):

• 82% of archives expect email holdings to grow significantly in the next 5 years.
• 4 million new emails and 100GB–1TB of new email collections are predicted per archive by 2026.
• By 2026, emails could account for more than 20% of public archives’ digital records collections.

According to the Background section of “Be Prepared,” this report is “a broadly-comprehensive guide to managing email records in state and territorial government” and presents CoSA’s thought leadership on archival preservation of emails.

CENTRALIZED GOVERNANCE WITH CAPSTONE

“Email management and preservation in government is primarily a challenge of governance,” says “Be Prepared.” Busy workers don’t categorize their emails as historically valuable (destined for the archives) or routine (OK to purge). They organize their email accounts according to personal preference. Therefore, CoSA recommends archivists participate in centralized governance of email accounts with Capstone, a position-based appraisal method pioneered by the National Archives and Records Administration (NARA) for emails of federal workers. With the Capstone method, archivists decide which email accounts go to the archives.

Here’s how Capstone works. Archivists analyze the work done by an email account owner in a high-level position or within a significant program. They estimate the likelihood that the owner sends and receives emails with historical value. As “Be Prepared” says, “If records from that program are classified as permanent, then so, too, should that account be classified as permanent.” Archivists then ask the IT professionals who support the email system to flag those email accounts for preservation.

When high-level officials such as the governor and department heads leave office, their email accounts are transferred to the archives. Accounts with a high probability—but not a certainty—of historical value, such as those of high-level advisors and policymakers, are appraised by archivists before transfer. In short, archivists achieve centralized governance of “accounts or mailboxes instead of messages,” says “Be Prepared.”

MIGRATION WITH NO SIGNAL LOSS

According to “Be Prepared,” email systems “are not typically optimized for traditional records management or digital preservation functions.” Archivists must migrate accounts to a digital repository system. This migration is to be accomplished with the preservation of as much of the original emails as possible (“no signal loss”). “Be Prepared” recommends that archivists do the following:

• Preserve all header information. Headers are the embedded metadata with information on senders, recipients, dates, and whether emails are part of threads.
• Preserve all message content with original formatting and any hyperlinks to related information outside the email.
• Preserve all attachments in the digital repository where they can be individually searched and indexed, and also preserve “the contextual linkages between the attached files and the email itself. …” Attachments are accessible from the original email, and vice versa.
• Preserve “any other contextualizing data whenever possible.”

Any digital repository system is to be compliant with the ISO Open Archival Information System (OAIS) reference model and have “tools which can handle the ingest of both emails and associated attachments as a single semantic unit.”

“Be Prepared” recommends that emails be converted from their native format to “open-source, stable formats. …” Electronic Mail Format (EML aka email) and MBOX Email Format provide “the most long-term flexibility.” A format to watch is the developing Email Archiving in PDF (EA-PDF). EA-PDF embeds original emails in their native format and their metadata within a PDF/A, the version of PDF for digital preservation.

“Be Prepared” suggests that a state or territorial archive begin its email preservation program with just two levels of workflow: a baseline workflow for most accounts and a premium workflow for important email accounts, such as those of the governor and department heads. In the premium workflow, archivists could incorporate natural language processing or AI for thorough arrangement and description.

It’s important just to get started. CoSA encourages archivists to start small and expand an email preservation program over time. A workflow “will be rough when it is first envisioned, and will be polished over time,” notes “Be Prepared,” but it is always designed to preserve all parts of emails.

A TEMPLATE FOR EMAIL PRESERVATION

“Be Prepared” also has a primer on email technology and a survey of current email governance in state and territorial governments. Research on email management tools is distilled into a guide with overviews and use cases. The table Key Email Management Workflow Steps lists roles and recommendations for email management, appraisal and selection, migration, archival processing, and access. An appendix has advocacy talking points for building collaborative relationships with other government agencies. Written by the leading association of state and territorial archives, “Be Prepared” is a thorough template for planning an email preservation program.

Other organizations wishing to follow up on CoSA’s work can study how local governments, often with fewer resources than state and territorial governments, can preserve their emails with historical value. For example, local governments may seek to preserve emails that document their responses to the COVID-19 pandemic. Another question for future research is possible uses of AI in email preservation workflows.

“Be Prepared” concludes that, despite the effects of other technologies such as social media, “[o]ne thing that remains true in government is the necessity for officials to communicate internally, as well as externally with members of the public; email has provided, and continues to offer, a simple, lightweight, and familiar platform for doing just that.” Archivists know that millions of emails are heading their way, and now they will, well, be prepared.