Transparency For the 21st Century: Leading Open Government
Ottawa, 23 March 2017
David S. Ferriero
Archivist of the United States
It is nice to be back in Ottawa. In my time as the Archivist of the United States, I have enjoyed a special relationship with not only Library and Archives Canada, but also with your Information Commission. And I would be remiss if I did not mention that I was appointed to my position upon the recommendation of the former American Ambassador to Canada, so you have a special place in my heart.
By way of setting the stage, let me tell you a little about the National Archives and Records Commission of the United States—who we are and what we do.
As the nation’s record keeper, we are responsible for guiding the creation, maintenance, and transfer for permanent retention and access the records of the Executive Branch agencies and departments, including the White House. That translates into about 275 entities from Cabinet level to small commissions. Similarly we house the records of the Supreme Court and provide courtesy storage and service for the records of both the Senate and House of Representatives. So, all three branches of our Government are represented in the more than 40 facilities across the United States which comprise my agency.
It was not until 1934 that the United States had a National Archives but we have been making up for lost time ever since! The records start with the Oaths of Allegiance signed by George Washington and his troops at Valley Forge and go all the way up to the Tweets that are being created at the White House as I am speaking.
That is a collection of some 13b pieces of paper, 43m photographs, miles and mile of film and video, and billions of electronic records.
Among those facilities are 14 Presidential Libraries documenting the administrations, family, and times of Presidents Herbert Hoover through President Barack Obama, just getting off the ground in Chicago. The libraries collect the papers, photos, video, electronic records, and artifacts of each Presidency.
To give you a sense of the rich history contained in those records, let me share a few examples relating to our shared history. (SLIDES)
Several additional units with the National Archives will be of special interest to this audience. (SLIDE)
Our Information Security Oversight Office (ISOO) which has the responsibility of monitoring the Executive Branch classification activities. ISOO’s annual report to the President tracks the number of classification authorities, original classification activity, declassification activity, and alerts the White House to the cost of classification activity. ISOO staffs an external Public Interest Declassification Board which reports directly to the President and advises on all things declassification related. Their report, “Transforming the Security Classification System” contains 14 recommendations which are in varying stages of implementation.
Our Office of Government Information Services (OGIS) serves as the Freedom of Information Act Ombudsman for the American public. Individuals who are having difficulty having their FOIA request dealt with can work with my staff to intercede on their behalf. OGIS provides training in remediation skills for the Executive Branch FOIA officers. OGIS also submits an annual report to both the White House and Congress.
The National Declassification Center (NDC) was established by Executive Order of the President at the end of 2009 and serves to review for release classified documents. With a motto of “Protect What We Must, Release What We Can” NDC works with the original classifiers to review their work for declass decisions.
Our Office of the Federal Register (OFR) publishes the Federal Register, the Government’s daily newspaper which provides, among other things, the text of proposed legislation with opportunities for the public to comment.
That’s a very quick overview of who we are and what we do, but I think you get a sense of the range of information activities for which we are responsible and the many intersections with an open government and transparency agenda.
At the National Archives, transparency and open government are fundamental to our work and clear in our mission: “We drive openness, cultivate public participation, and strengthen our nation’s democracy through public access to high-value government records.” Public access to government records has been at the core of the mission of the agency since we opened our research rooms to the public in the 1930s.
Each new Presidential administration comes into office with aspirations for enhancing the quality of life for all Americans by improving how government works. The Obama administration’s Big Hairy Audacious Goal—beyond Health Care—was Open Government.
On his first day in office he issued his Memorandum on Transparency and Open Government committing to an unprecedented level of openness in Government to ensure the public trust and establish a system of transparency, public participation, and collaboration. And in December of 2009, an Open Government Directive charged each Executive Branch Agency to create a more open government by publishing government information online, improving the quality of government information, creating a culture of open government, and formulating an Open Government Plan.
The timing was perfect for a new Archivist beginning his tenure. I was wooed away from what now looks like a cushy job at the New York Public Library because of the new administration’s belief that the National Archives could and should play an important role in the Open Government Initiative and it was that belief that excited me about signing on in November of 2009 just a month before the Directive was issued.
We issued our first Open Government Plan in April 2010 and our fourth and most recent in September of 2016. (SLIDES)
A romp through the early reports gives you a good feel for the agency’s cultural changes and growth, the influence of new technological tools, proof that the administration was right on the mark that the National Archives could be a contributor. And, in fact, a leader.
The First Open Government Plan covered 2010 through 2012 and we set out 70 open government commitments, including expanding employee and public engagement and dramatically expanding our social media presence. We established good records management as “the backbone of open government” because the long-term success of the Open Government Initiative hinges on the ability of each Federal Agency to effectively manage its records.
I launched my own blog, AOTUS, to communicate directly about our initiatives. More than 250 posts with 500k views to date.
It was also an opportunity to rethink the organization to be better prepared to deliver on the promises of open government and transparency. The process enabled us to use social media tools to engage an agency of 3000 people across the country in creating a transformation agenda which committed to working as one agency, to embrace the primacy of electronic information, to foster a culture of leadership at all levels—a great place to work, and to open our organizational boundaries to learn from others and share what we know.
Our Second Open Government Plan, 2012-2014, set forth another 30 initiatives, including a “citizen archivist” program. On his first day in office, President Obama told his senior staff: “Our commitment to openness means more than simply informing the American people about how decisions are made. It means recognizing that Government does not have all the answers, and that public officials need to draw on what citizens know.” We took that message to heart in looking for ways to engage the public in our work. We created the Citizen Archivist Dashboard to crowdsource activities such as tagging of photos, transcribing records in cursive or a foreign language, and scanning missions on specific topics.
The Second Plan also strengthened Open Government by improving coordination and collaboration among agencies and effectively streamlining the declassification process, with a goal of never getting into the backlog situation which resulted in the President’s Directive establishing the National Declassification Center.
This plan also triggered the crafting of the National Archives Strategic Plan committing to making access happen, connecting with our customers, maximizing our value to the nation, and building our future through our staff. Make access happen includes a goal of digitizing everything we own. A goal which scared the staff!
By 2014 when we created our Third Open Government Plan covering 2014 through 2016, transparency, participation, and collaboration were already at the core of all we do. By this time more than 160 external projects on more than 15 social media platforms were in place.
Our flagship initiative focused on Innovation—including significant improvements in our online catalog—creating a public API, introducing crowdsourcing, and mobile optimization of the site. We also created an Innovation Hub—in both physical and virtual space—to experiment and find innovative solutions, including development of new approaches to enhance digital access.
Which brings me to our most recent work, the Fourth Open Government Plan which covers 2016 through 2018. This plan serves as a public road map of our commitments to strengthen transparency and open government. Published this past September, the plan includes 50 new actions to which we have committed in order to strengthen open government through all business lines with the agency.
While we were concerned about impact of the end of the Obama administration on the level of interest of the staff, I was pleased to see the growth in excitement and enthusiasm, both internally and externally, as we developed the plan with extensive feedback and consultation from our staff and with the public, especially the civil society community. We have become a Federal leader in the consultation and development process for our plans.
We held more than 20 internal brainstorming sessions and briefings, including holding our first Open Government Town Hall, which was webcast for staff participation across the country. This was an important event so that staff at all levels knew that their voice mattered—that we were asking them for their ideas on what we could and should do better. I led Agency senior executives in a robust discussion and heard from our staff their ideas on making improvements to better serve the public.
We held our first Open Government Webinar for the public and external stakeholders. We provided an overview of our open government efforts in a variety of topic areas and gave stakeholders an opportunity to share their thoughts on what we should be doing.
The dialog resulted in more than 180 ideas, comments, and suggestions for inclusion in the plan. Most importantly, we published the plan on Github, allowing us to publish the code for the plan, transparently showing all the changes and edits we made and allowed us to receive ongoing suggestions, including specific language updates to the plan. Publishing the code on Githab also enabled our General Services Administration to “fork” the effort and re-use for publishing their own open government plan. Our efforts and collaboration with another agency were lauded by civil society organizations as efforts that should be replicated by others.
Our plan has ten flagship initiatives to increase access to our records through innovative efforts. These initiatives are significant cross-agency efforts that work together to achieve the agency’s strategic goals and directly impact on our goal of making access happen.
The initiatives:
The Innovation Hub is new collaborative space for work on enhancing access to records through Wikipedia edit-a-thons and hackathons. A public scanning room encourages researchers, students, and educations to come and digitize our records at no cost. They get a copy of what they need and we get a copy to fulfill my dream of digitizing everything we own.
The History Hub at history.gov is transforming how we answer questions from the public and we are working to build a support community for researchers, citizen historians, and archival professionals. History Hub offers several tools like discussion boards, blogs, and community pages to bring together a community to provide the public with unprecedented access to history, experts and government information. It is a place for the public to share expertise, collaborate, find people based on their experience and interests, and get help with their research from experts at the National Archives facilities across the country. We will soon be expanding to include the Library of Congress and the Smithsonian Institution.
Our new Social Media Strategy was developed in consultation with internal and external stakeholders and published on Github where the strategy is available for reuse by other agencies and cultural institutions, so check it out. The four goals of this strategy: Tell Great Stories, Deepen Engagement, Grow Our Audience, and Cultivate a Social Media Community. Access and transparency are at the core of our work. With the explosion of digital choices, audience needs have changed and their criteria for following cultural organizations have matured. We need to provide exceptional content to stand out—even if it means reaching beyond our comfort zone and trying new approaches.
What started as an experiment in crowdsourcing, our Citizen Archivist Dashboard has become core to the way we do business and the opportunities for the public to improve access grows—the ability to transcribe and contribute metadata directly through the catalog, for example. More than 500k enhancements by citizen contributors under our belt. And the recruitment of “community managers” will help us develop engaged communities around our records.
Ways into the vast content of government records have frustrated the public forever. New ways of tapping the crowd to create and share their own finding aids and harnessing those successes for other researchers.
Transforming the agency’s front door, archives.gov, which already serves more than 3m people a year to provide even more transparency and openness is the rationale behind our recent move to Drupal and the cloud, using corporate user-centered practices into a redesign and iteratively improve or build from scratch new tools based on user needs and data analysis.
The National Archives catalog now contains more than 24m digital objects, including digitized and born digital records. More than 94% of the records of the National Archives are described in catalog. We are hard at work on improved the catalog API and leveraging it to develop new tools, metrics, and embed digitized records on Archives.gov.
Aligning External Standards with current practice to enable users to be successful with commonly accepted ways of organizing and describing information.
Providing the contextual framework for our content in an international collaborative venture will improve the economy and quality of processing and description.
Our plan to digitize everything will depend upon partnerships, crowdsourcing, agency transfer of digitized rather than paper records, internal digitization as part of a business process, and our own Digitization Labs. Lots of planning underway.
Our four Open Government Plans have not only informed my agency’s fulfillment of our mission, but our work has captured the attention of the White House Office of Science and Technology which oversees the Open Government initiative. We have, in fact, provided a staff person to that office who has provided subject expertise to the Administration on policy issues related to open, participatory, and collaborative government, including Freedom of Information Act policy and modernization efforts. This Archives staffer has also coordinated the development and implementation of the U.S. Open Government National Action Plans as part of the Open Government Partnership. (SLIDE)
The Open Government Partnership is an international initiative that commits governments to promote transparency, empower citizen, fight corruption, and harness new technologies to strengthen governance. Launched in the Fall of 2011 with eight founding governments, there are now 75 participating countries who have made more than 2500 commitments to make their governments more open and accountable. Both the United States and Canada are participants.
The most recent U.S. Open Government National Action Plan has the National Archives fingerprints all over it! We hosted an interagency event with the White House to garner public input into the development of the plan, which featured presentations from government and civil society leadership. We provided staff support to an interagency team for six months in the development of the plan and participated as part of the U.S. delegation at the Open Government Partnership Global Summit.
Most importantly, and the thing of which I am most proud, is the fact that five of the 11 National Action Plan “Access to Information” commitments and one of the four “Public Participation” commitments are owned by the National Archives: (SLIDE)
Improving Management of Government Records
As I have said the backbone of a transparent and accountable government is strong records management. Our Managing Government Records Directive required agencies to manage all of their email in electronic form by the end of 2016 in preparation for an electronic record only environment by the end of 2019. To support these requirements we commit to increasing transparency in managing email. We will release a public dataset of positions of government officials whose email will come to the National Archives for permanent retention under the Capstone approach (senior level agency officials). This dataset will increase facilitate public participation in the ongoing dialogue over records that document key actions, policies, and decision of the Federal Government.
We will report on agency progress in managing email allowing stakeholders to track progress on agencies’ email management efforts.
And we will improve the Records Control Schedule Repository by working with the civil society community to improve access to the data contained in the repository.
Modernizing Implementation of the Freedom of Information Act
We commit to working collaboratively with the Department of Justice, agencies, and the public to expand the services offered on FOIA.gov to exploit technology to better serve the stakeholder communities. We are exploring a guided request tool approach, online tracking of request status, simplified reporting methods for agencies, improved FOIA contact information, and tools that will enhance the public’s ability to locate already posted information.
We are testing, with Justice and seven agencies, proactive disclosure by posting FOIA-released records online. This pilot will address the costs associated with such a policy, effect on staff time to process, effect on interactions with stakeholders, and the justification for exceptions to such a policy.
We are working with Administration to issue guidance and best practices for improving agency FOIA websites, including a template for key elements to encourage all agencies to update their FOIA websites to be consistent, informative, and user-friendly.
FOIA continues to be a great mystery to the most of the American public, so we are developing tools to teach students about FOIA, drawing on real-world examples to foster democracy and explain how the public can use FOIA to learn more about the government’s actions.
Streamlining the Declassification Process
Earlier I mentioned the work of the Public Interest Declassification Board. Their recommendations to the President are reflected in the promises in this commitment.
The interagency Classification Reform Committee will develop a plan to implement technological tools to help automate declassification review, expanding on the work the CIA and the National Archives have been doing.
One of the things that really bogs down the declassification process has been the multiple agency equity issue. To address this, the Classification Reform Committee is piloting a topic-based interagency declassification guide. This is declassification based on a topic or even to enable trained interagency staff to review this information where it resides, rather than referring the classified information to multiple agencies, resulting in lengthy delays.
The National Declassification Center is establishing a special systematic declassification review program for previously reviewed and exempted historical records that were accessioned to the National Archives and reviewed prior to the creation of the National Declassification Center in 2010.
The CIA led an interagency project declassify no-longer sensitive Presidential Daily Briefs from the Nixon, Ford, Kennedy, and John administrations which are now posted on the CIA website.
Implementing the Controlled Unclassified Information (CUI) Program
There is a fair amount of information created that does not meet the threshold for classification but must be safeguarded and dissemination controls established which are consistent with law, regulation, and government-wide policies. The lack of consistency across the Government is being address by ISOO which has issued guidance, established a phased implementation schedule, trained agency staff, and created an enhanced CUI registry that designates what information falls under the program.
Develop a Machine Readable Government Organizational Chart
The United States Government Manual, published by the National Archives, has provided access to agency organizational information and charts since the 1940s. To facilitate access to government agencies, the General Services Administration will work with our Office of the Federal Register to capture agency organizational directories as raw data in a consistent format across the Government. Documentation for this format will be made available so that other government bodies, including local governments, can also public their office names, organizational structure, and contact information as standardized open data. Making this data public and consistently available across the Federal Government will help the public find the offices and officials that serve them in a simple and straightforward manner.
So, I am pleased to report that the Administration‘s belief that the National Archives could play a role in the Open Government Initiative has panned out. Panned out in ways that we never could have predicted.
Let me conclude with a few lessons learned in the process.
The impact on the staff had been rewarding. It is a passionate staff who cares about history, the records, and connecting people with the information they need. To have those competencies recognized and valued by the White House in this process has been a terrific morale booster.
Demonstrating our competency, tech savvy, and willingness to work hard showed a side of the National Archives that our sister Agencies had not seen before.
The support and enthusiasm of the civil society community has been inspirational. Their demands for openness and transparency have been well received and valued by the National Archives.
The engagement of the entire staff in the planning process has been key. This is an agency wide initiative and everyone has a stake in our success. Providing opportunities for input have been very important.
The emergence of new technologies in the life of Open Government have transformed the process and the plans. New opportunities not foreseen in the first plan are now at hand and inspire us to think about the future.
The need to be more pro-active with our international counterparts has become clear, both in sharing our experience and learning from theirs.
Finally, it just warms the cockles of the Archivist’s heart to be able to say that we have assumed a leadership role in Open Government for the United States!