Honoring the Past, Imagining the Future:
The View from the National Archives and Records Administration
JOHN W. CARLIN
Archivist of the United States
Address to the National Association of Government Archives and Records Administrators
July 15, 1999
Less than two weeks ago, I participated in a dramatic event that should warm the hearts of all of us who care about records. It took place in the magnificent rotunda of the National Archives Building in Washington. More than a hundred prominent citizens had assembledgovernment officials, officers of corporations, heads of foundations, and representatives of the Congress. Music played; in came a uniformed color guard from the Armed Services; and not far behind, the President of the United States and the First Lady.
They were all there to celebrate America's great Charters of Freedomthe Declaration of Independence, the U.S. Constitution, and the Bill of Rightsand to kickoff publicly the campaign for NARA's "Charters of Freedom Project." Already, the Congress and the Administration are giving us money to renovate the building that has been the Charters' home for nearly fifty years. The Pew Foundation has added funds to help finance a re-encasement of the Charters so that we can continue to display them safely for future generations. And on this occasion, the AT&T Corporation pledged a million dollars to help us preserve the Charters in a meaningful educational setting. If we raise enough additional money, visitors in the twenty-first century will not only see these documents; they'll also understand, from exhibits, theater programs, and educational and research opportunities, what the Charters mean in American life.
And I was really struck by the fact that all these prominent people had come there to join with us to ensure preservation and access for these three records. And I thought, now if only I could get everyone also to recognize the value of the other millions of records in our custody!
For as Emily Mitchell recently wrote about the National Archives in Time magazine, "On one side of the building are the grand documents of democracy . . . . On the other side are the commonplace but invaluable records of the 272 million people who make up that democracy." 1. She was talking of census records dating back to the eighteenth century, of Civil War pension records from the mid-nineteenth century, and of immigration records into the twentieth century. But additionally, I was thinking of many records outside our Washington facilities, such as the thousands of records of twentieth century military service that I've just declared of historical value for permanent retention in the National Archives. And I was thinking of all the thousands of records still in the custody of agencies, and the thousands of new records they are generating every day, which also are part of our concern.
Probably we're doing all right with the great documents in this country. But in records management and archival preservation more generallyare we? Or was all the pomp and circumstance in the rotunda that day simply tending to cover up the sinister sound of unnoticed events: the slow cracking and flaking of paper, the fading of unstable color dyes in films and photographs, and the disappearance of digitized data from electronic disks and tapes, not to mention the pressing of "delete" buttons in government offices and the rattling of shredding machines.
Happily, a more sober and less frightening view is available from a historical perspective. We can take some comfort from certain comparative statistics. In the 1960s, a study by the archivist Ernst Posner found that only 30 of the 50 states had full state-government archival programs, twelve had partial programs, and eight had none in state government. In the 1990s, a study by the national Council of State Historical Records Coordinators reported that all states but one had full archival operations, and every state plus the District of Columbia and the territorial jurisdictions of the U.S. was providing some degree of care for documentation of at least the activities of state government.2
It is progress in itself that there now exists a Council of State Historical Records Coordinators to make such a survey. And it is progress that almost every state has an active Historical Records Advisory Board. As you know, such boards help the National Historical Publications and Records Commission, which NARA administers, make grants to local archival institutions and records management programs, which also have been growing over the years. I have no doubt that this federal/state/local partnership has fostered advances in records care all across the country. And I call on all of you to continue to support the collaboration of NHPRC, state boards, and local institutions.
Nonetheless, it's one thing to say that government archival programs are growing, and it's another to say that they have grown as fast as governments and the records they generate. And it's one thing to say that we have more records management programs than ever, and another to say that agencies are paying attention to them. All you have to do to become concerned about that is to keep reading the papers.
What newspaper in the country has not recently carried reports of the possible loss of records documenting entitlements to millions of dollars that may be owed to Native Americans? The Interior Department, and the Bureau of Indian Affairs within it, have come in for intense recent criticism in the Congress and in court for allegedly being unable to produce the relevant documents.
On another front, the Associated Press recently issued nationwide a story headlined, "Alcatraz Inmates' Records Elusive." Researchers complained that records on Al Capone, Machine Gun Kelly, the Birdman of Alcatraz, and other Alcatraz Prison inmates seemed to be missing. Our initial investigation concurred that files appeared incomplete through possible losses while the records were in the custody of the Federal Bureau of Prisons.
Another example is a story carried by the Washington Post under the headline, "Gulf War Logs on Chemicals Reported Lost in Office Move." It reported a charge by a Pentagon inspector general that certain chemical weapons logs from the Persian Gulf War were missingrecords that some veterans' representatives speculated might contain information on harmful chemical exposures. The report found no evidence of a conspiracy to destroy or conceal the logs, but speculated that Army personnel destroyed them as part of moving an office.
My intention in citing such examples is not to single out particular agencies for criticism. NARA has its own problems, and one of them is that we've sometimes been slow to provide the records management help that agencies need, particularly for managing electronic records. But my real point is that records management matters. In the examples I've given, keeping the right records affects people's money, their health, and their history. And however far we've come in making the case for records care in our society, we haven't come far enough.
For how many more cases will come to light about records that the Government no longer has? And how will we preserve and provide access to all the new kinds of records in the age of electronic information? Recently I listened to a discussion of the need to deal with electronic texts, images, photos, videos, audios, e-mail messages, compound records, Web sites, geographic information systems, and something called records in "hexidecimal form."
These specifics and the electronic record challenge are now so well known that even the press is taking interest. Last year I published an article in the Washington Post on the problems that new technologies pose for record keeping. Subsequently there have been substantial reports on the subject in the New Yorker, Newsweek, the New York Times, the Wall Street Journal, and at least one network news program. I really loved a quote attributed by the Associated Press to Jim Henderson, Maine's state archivist, who I'm grateful to say has worked with NARA on these issues. Asked about how electronic records differ from traditional kinds, Jim was quoted as saying, "You're dependent on the machine . . . The machines are constantly changing, and the software is constantly changing and the media on which the information is stored is constantly changing. In the good old days, when man was deciphering hieroglyphics on rocks, at least the rocks just stayed right there."3
The Wall Street Journal declared that "the National Archives has a monumental challenge: making sure government records are preservedand accessible" in the electronic age. Ironically, on the same page with the "electronic challenge" story was some company's advertisement showing several business people studying a problem over the caption, "With enough crumpled paper, you can solve anything."
Well, records managers don't crumple paper, but we are getting closer to some real solutions. A lot has happened on the records front since I met with you at NAGARA's conference last year. And many of you worked with us to make it happen.
Some of you served on NARA's Electronic Records Work Group, composed of staff members from NARA and other agencies, and supported by outside expert consultants. Last year, as you're aware, the group provided recommendations, on the basis of which we at NARA have subsequently issued guidance to Federal agencies on scheduling how long to keep electronic copies of certain records that remain on an e-mail or word-processing system after a record keeping copy has been produced. We also developed changes to general records schedules that authorize the disposal of certain administrative records, regardless of physical format. And we are making progress on developing a new general records schedule for administrative records documenting the management of information technology; that is, administrative records generated in IT shops. Already this spring, this IT records schedule has had two rounds of review by the agencies, and when we have finished studying the comments, we will publish the resulting version in the Federal Register for public comment. So far the agency comments for the most part indicate that we are on track in the schedule's coverage of records and the retention periods for these records.
Additionally, with input from NARA staff, the Department of Defense developed a set of baseline requirements for the management of its electronic records, and issued criteria for the design of computer software for use in electronic records management. After independent evaluation, we endorsed this DoD standard as consistent with the Federal Records Act and of potential usefulness to other Federal agencies. The standard does not answer all pertinent questions nor preclude other approaches, but it does provide at least a starting point for agencies that want to begin implementing electronic record keeping now. We are at this time reviewing DoD's certification process for software meeting baseline requirements. And we are working with DoD and other Federal agencies on several other technical projects of potential value to government agencies at all levels in dealing with electronic records.
Additionally, we've now launched a "Fast Track Guidance Development Project" to identify "best practices" currently available and to provide guidance quickly on electronic records issues that urgently confront Federal record keepers nowguidance they can use while work goes forward on developing more complete and longer-term solutions. "Fast Track" is a collaborative project involving persons with relevant expertise from NARA's staff, individuals from other Federal agencies with expertise in records management, systems development, and information technology, and expert consultants from outside the Federal Government.
Also, and perhaps most far-reaching of all, we've launched an effort addressing how records are scheduled and appraised in the Federal Government. The responsibility for approving the disposition of records is perhaps the most critical statutory responsibility held by the Archivist of the United States because it determines how long records must be kept to protect individual rights, provide accountability in government, and document the national experience. Therefore, the scheduling and appraisal process that we use to carry out this responsibility is central to NARA's mission. It is also an important tool for managing records throughout their life cycle, and it is a process that our Strategic Plan identifies as flawed and in need of a major overhaul.
The scheduling process developed during the twentieth century and currently used by the Federal Government is based primarily on paper-based record keeping systems as they were used at mid-century. The reality at the end of the twentieth century is that most records are created electronically and may be maintained in a variety of media. Each of the media typespaper, microform, film, electronichas advantages and disadvantages as a recording medium. And agencies need to know how to manage the disposition of all documentation they create, regardless of media, in light of current record keeping practices.
Therefore, the goal of this project is to define what should be the Federal Government's policies on determining the disposition of Federal records, the processes that will best implement those policies, and the tools that are needed to support the revised policies and processes. We must answer a number of basic policy questions about Federal documentation, the goals and purposes of scheduling, the appraisal criteria to be used in determining appropriate retentions, and the respective roles of NARA, Federal agencies, and the public in achieving the goals and making the process work effectively. We must make the scheduling process more effective and efficient, and decrease the time that it takes to get schedules approved. And we must use automation to support the scheduling process as part of managing records throughout their life cycle. In addition, I would like to thank Mike Miller for taking temporary leave from his administrative and management responsibilities in order to head this important project, this being the first in a series of efforts to overhaul all the processes used to manage records throughout their life cycle.
Also exciting, with new funds in our current budget, we are in the process of adding thirteen professionals, all with experience in all media including electronic, to our records management staff. This will allow us to expand assistance to Federal agency headquarters and field offices through our new Targeted Assistance Program. And if we get the funding we requested for next fiscal year, we will add another thirteen professionals, bringing our total new positions for this effort to twenty-six. I know it has often been said that nothing is really new, and that certainly applies here. This records management initiative is patterned after one that Deputy Archivist Lew Bellardo developed, with NHPRC support, in Kentucky, where he produced a very successful statewide effort to help local units of government dramatically improve their records management program. Lew has worked very closely with Mike Miller, Diane LeBlanc, Kent Carter, and others to get this program off to a good start, and I am very appreciative and excited about what we can do.
Let me also comment on a couple of other ways in which we are implementing our Strategic Plan. As we have communicated to you in various ways and times, we are taking our records center program reimbursable. Many, if not most of you, either have been involved in similar programs, or, as Federal officials, you soon will be. One very significant bi-product of that effort has been our creation, for the first time, of comprehensive standards for the storage of records still in the custody of the agency. They will soon be finalized, and I want to take this opportunity to thank Roy Turnbaugh, Bruce Dearstyne, and all of you who assisted us with your individual as well as organizational comments on these standards. I also want to say that I am very pleased that David Weinberg has accepted the leadership responsibility for NARA's reimbursable program.
Another area about which we have exchanged much information is our effort to deal with space issues from both a quantitative and qualitative perspective. Recently, I announced that, in addition to the renovation of Archives I and the securing of appropriate space for our military personnel records in St. Louis, our next focus will be on replacing our Atlanta archival facility. We are in the early planning stage in Atlanta, where one possibility we very much like would be to build a facility adjacent to a new Georgia State Archives (also in the planning stage), and, in the process, develop some synergy and efficiency that could benefit both researchers and staff.
And speaking of hopes and dreams, something else is happening of potential long-term importance to government archives and records administrators. We mayand I emphasize "may" because we're still in the R & D stagebut we may have in sight a workable way to archive electronic records in a comprehensive system providing both preservation and access for all data types without dependence on particular software or hardware.
For some time now, NARA has been able to accession small numbers of large electronic databases. But the Government is increasingly generating large numbers of small files, such as e-mail messages. Facing the fact that available technologies are not adequate for the task of archiving them, we joined the National Partnership for Advanced Computational Infrastructure to see if advanced computer science could provide a solution. I recently learned of a prototype system developed by our partners at the San Diego Supercomputer Center, a national laboratory for computational science and engineering at the University of California, San Diego. The prototype has been able to "ingest"their worda million e-mail messages in just two days.
That is progress. But additionally, research being done for us at the Supercomputer Center, and also at the Army Research Laboratory, gives us hope that an Electronic Records Archive can be built to preserve any kind of electronic record in a format that frees it from the computer system in which it was created and will enable us to meet reference requests for it using a variety of tools available today and advanced technologies that will be developed for tomorrow. And access will be expedited by combining this system with the Archival Research Catalog we are developing to describe all bodies of records in our nationwide holdings. Furthermore, we asked our partners to develop a scalable solution. That is, NARA needs a system that will scale upward, so we can deal with accessions of millions of records, but the approach we are taking also promises to scale downward, so it will be useful for smaller archives as well.
Whether all this works or doesn't is yet to be seen. But let me conclude by saying that sooner or later something of the kind must work. It must because successful records management in the twenty-first century requires it. And because our society requires successful record management and archival administration.
Sometimes you have to get beyond your own area of activity to understand what that really means. Everything we are doing these days at the National Archives and Records Administration follows a Strategic Plan. The heart of that plan is a mission statement that commits us to ensure ready access to essential evidence, documenting the rights, identities, and entitlements of citizens; the actions for which Federal officials are responsible; and the historical experience of our nation. But if you really want to know how important that is, ask the ethnic Albanians now returning to their ruined communities in Kosovo. During the conflict there, CNN reported that the horrors of ethnic cleansing had come to include the deliberate destruction of public records as a means of erasing the identities and the culture of Albanian Kosovars.
I think back now to that event in the National Archives rotunda that I described at the beginning of this address. And I think how particularly important records are in a democracy. Our government, and our way of life, is not based on the divine right of kings, the hereditary privileges of elites, or the enforcement of deference to dictators. It is based on those pieces of paper, those Charters of Freedom, the Declaration that asserted our independence, the Constitution that created our government, and the Bill of Rights that established our liberties. And I remember something that Senator Trent Lott recently observed about them. Abroad, we tend to go see a nation's crown jewels as an expression of its glory. Here, our national crown jewels are these pieces of paper enshrined and displayed to visitors in the National Archives.
Our democracy depends on them and on millions of other records in the care of government archives at all levels. For in this country, records define all of our governments, document all of our identities, establish all of our entitlements, and enable us to hold accountable those to whom we entrust office, federal, state, and local. Collectively, we have an awesome responsibility. And we face more challenges than ever to meeting it. But even in dealing with electronic records, I believe we are making progress. In the long run, I think we can all succeed. And I know, for the sake of our democratic society, that we must. Thank you very much.
1. Emily Mitchell, "A Visit to the National Archives, The American People's Library," Time, Apr. 19, 1999, p.67. Return to text.
2. Ernst Posner, American State Archives (Chicago: The University of Chicago Press, 1964), and Victoria Irons Walch, Recognizing Leadership and Partnership, a Report on the Condition of Historical Records in the States and Efforts to Ensure Their Preservation and Use, with a focus on State Historical Records Advisory Boards and State Archives and Records Management Programs (Iowa City: Council of State Historical Records Coordinators, 1993). Return to text.
3. Associated Press, "Archives braces for onslaught," Boston Globe, Dec. 26, 1997, p. A-13. Return to text.