NATIONAL ARCHIVES AND RECORDS ADMINISTRATION
RECORDS SCHEDULING AND APPRAISAL SERVICES
CUSTOMER SATISFACTION SURVEY REPORTJULY 2005
- Research Methodology
- Results Statistics
- Next Steps
In August, 2004, the National Archives and Records Administration (NARA) conducted a survey of agency Federal Records Officers (FROs) to elicit feedback regarding these customers' satisfaction levels with its scheduling and appraisal services. The survey inquired about each FRO's overall satisfaction level as well as satisfaction with the following aspects of NARA's scheduling services: timeliness, utility of guidance products, and adequacy of communication. NARA distributed 228 questionnaires and received 115 completed surveys for a response rate of 50%.
Federal Records Officers reporting that they are satisfied or very satisfied with NARA appraisal and scheduling services is 56.5%. Almost a quarter (22.6%) of FROs report that they are neutral about these services and 20.9% report that they are either dissatisfied or very dissatisfied.
Survey respondents express the highest satisfaction ratings with the working relationship they have with their appraisal archivist and the response they receive when they contact NARA for assistance. About four out of five survey respondents (80.9%) agree or strongly agree that they have a good working relationship with their NARA appraisal archivist. The vast majority of all survey respondents (75.7%) also agree or strongly agree that they receive a response that meets their needs when they contact NARA for assistance.
Respondents give the lowest satisfaction ratings to the timeliness of the appraisal process and to being kept informed by NARA about the progress of records schedules throughout the approval process. Only one out of every four respondents (25.2%) agree or strongly agree that the time it takes to approve a records schedule is satisfactory. Just half (49.5%) agree or strongly agree that they are kept informed by NARA about the progress of their schedules. There also appears to be a direct correlation between a respondent's overall satisfaction level and whether they are kept informed of the status of their schedules.
The responses of survey participants to the four questions about NARA guidance fell in the mid-range of ratings and were roughly equivalent, ranging from 60.8% to 64.3% of all respondents agreeing or strongly agreeing that NARA scheduling and appraisal guidance is produced in a timely fashion, comprehensive enough, easy to work with, and consistently interpreted by NARA staff.
NARA is committed to improving its scheduling and appraisal services and is working to increase the overall satisfaction level of its customers by focusing on the three areas of special concern identified by this survey's respondents: the timeliness of the appraisal process, keeping agencies informed about the progress of records schedules throughout the approval process, and the utility of its guidance products.
This survey was designed to help the National Archives and Records Administration (NARA) increase customer satisfaction with its scheduling and appraisal services. It was conducted to establish NARA's baseline scheduling and appraisal services customer satisfaction level and to serve as a tool to elicit decision-making information to help NARA improve its services. The survey asked Federal Records Officers, who oversee each agency's scheduling efforts, how they feel NARA is doing in this area, why they feel the way they do about NARA's level of service, and what is extremely important to them so NARA will know where to invest its resources to make effective changes to its business processes.
This survey's questionnaire was designed to gather information from Federal Records Officers about their overall satisfaction level regarding NARA's scheduling and appraisal services and also included eight core questions about three specific aspects of this service: timeliness, quality of communication, and utility of guidance products. Each of these eight questions allowed respondents to indicate whether they strongly agree, agree, are neutral about, disagree, or strongly disagree with a particular statement (i.e., Survey Question). Respondents also had the option to indicate when they felt one of the eight survey questions was not applicable.
In addition to the core questions, this survey instrument includes three demographic questions and a narrative section. The demographic questions asked for respondents' agency type and size as well as the number of schedules a respondent's agency had submitted to NARA during the past 12 months. In the narrative section, survey participants were given an opportunity to submit comments about what they felt was the "most important thing [NARA] could do to improve our scheduling and appraisal services."
NARA directed this survey to agency Federal Records Officers or the FRO's designated point of contact because an agency's appraisal and scheduling activities are primarily overseen by this staff. Surveying only FROs, rather than all Federal agency records management or program staff involved in scheduling activities, allowed NARA to ask questions regarding satisfaction levels with scheduling and appraisal guidance products that might have very little meaning to agency staff who do not receive notifications regarding NARA policy and procedure changes and have little experience with the General Records Schedules (GRS). Questionnaire recipients were encouraged to confer with other records management staff in the agency as needed in order to determine how best to answer the questions in the survey.
NARA sent one questionnaire to each agency surveyed. This leveled the playing field among agencies with different records management staff sizes. This survey defined an agency as an office with department-wide responsibilities in a Cabinet-level department, a bureau or agency within a Cabinet-level department, or an independent agency having an FRO.
Respondents completed the questionnaire on-line. It was conducted over a four-week time span from August 9 to September 3, 2004. The target audience received a link to the questionnaire via e-mail. The survey pool received a pre-survey notification via e-mail one week before the questionnaire was distributed. Because the survey maintained respondents' anonymity and did not track who was submitting completed questionnaires, everyone in the target audience received weekly follow-up e-mail notices thereafter.
NARA plans to issue a scheduling and appraisal services customer satisfaction survey at least once every two years. Conducting this survey regularly will allow NARA to track customer satisfaction rates for scheduling and appraisal services over time.
NARA received 115 completed questionnaires out of 228 distributed to the target audience for a response rate of 50%. The total number of survey respondents for questions 1-12 in the tables below is 115. The bold number represents the number of responses for a particular category. The second number in each box shows the percentage of survey participants who chose that response.
Survey participants were asked to respond to the following statements and questions:
|Very Satisfied||Satisfied||Neutral||Dissatisfied||Very Dissatisfied|
|9. How satisfied are you with NARA scheduling and appraisal services?||18 |
|1,000 or less||1,001-5,000||5,001-20,000||20,001-50,000||50,001-100,000||Over 100,000||I do not know.|
|11. Please indicate the number of full-time employees in your agency.||26 |
|0||1-5||6-10||11 or more||I do not know|
|12. How many SF 115s has your agency submitted to NARA during the past 12 months?||35 |
NARA received responses from all agency types. About 18% of respondents are in offices with department-wide responsibilities in a Cabinet-level department, 48.7% of respondents are in bureaus or agencies within a Cabinet-level department, and 33.0% are in independent agencies.
The agency size of the survey's respondents varies. About half of the survey respondents (52.2%) belong to agencies with 5,000 or fewer full-time employees. Regarding the others, 17.4% belong to agencies with between 5,001 and 20,000 full-time employees, and 20.0% belong to agencies with more than 20,000 full-time employees. One in ten respondents (10.4%) does not know the size of their agency.
The highest number of respondents (40.0%) submitted from one to five schedules to NARA during the past 12 months. A quarter of respondents (25.2%) submitted six or more. Only 4.4% of respondents indicated that they did not know how many schedules their agency had submitted during the past 12 months.
Almost a third of this survey's respondents (30.4%) indicate that they had submitted no schedules (i.e., SF 115s) to NARA during the past 12 months. However, judging by the comments submitted by this demographic, these respondents do have experience with the schedule approval process as well as with NARA's scheduling and appraisal guidance products. It is conceivable that an agency might be involved in the schedule approval process for a schedule or schedules older than 12 months.
Chart 1 shows how many survey participants responded to questions one through eight.
The following charts show how survey participants responded to questions one through eight according to their overall satisfaction level.
Chart 5 represents the percentage of overall satisfaction levels by the number of schedules (i.e., SF 115s) submitted to NARA during the past 12 months.
This section of the report discusses the results of the overall customer satisfaction survey question and the core eight survey questions as they relate to three aspects of scheduling and appraisal services: timeliness, adequacy of communication, and utility of guidance products. It analyzes many of the statistics by a respondent's overall satisfaction level. This report divides respondents into three categories according to how a survey participant responded to Question 9 ("How satisfied are you with NARA scheduling and appraisal services?"). For the purposes of this report, "satisfied" FROs comprise survey participants who have either a satisfied or very satisfied overall satisfaction level, "neutral" FROs have an overall satisfaction level of neutral, and "dissatisfied" FROs have overall satisfaction levels of dissatisfied or very dissatisfied.
Federal Records Officers reporting that they are satisfied or very satisfied with NARA appraisal and scheduling services is 56.5%. A total of 65 respondents characterize themselves as being very satisfied or satisfied with NARA scheduling and appraisal services. Twenty-six respondents indicate they are neutral about these services (22.6%) and twenty-four (20.9%) characterized themselves as being either dissatisfied or very dissatisfied overall.
The satisfaction rate of FROs who report that they submitted one or more schedules to NARA during the last 12 months is almost exactly the same as the satisfaction rate of all customers. Fifty-six percent of these customers state that they are satisfied or very satisfied with NARA appraisal and scheduling services (42 out of 80 total). The satisfaction level of respondents who indicate they had submitted no schedules to NARA during the last 12 months is relatively evenly distributed and their responses to the eight survey questions match the overall general trends of the total respondents; their responses are not grouped together under any specific category. (See Chart 5 above.)
Timeliness of Records Schedule Approval Process
Survey Question 1: The time it takes to approve a records schedule is satisfactory.
Federal Records Officers express the highest level of concern with the amount of time it takes to approve a records schedule. Overall, 25.2% of respondents agree or strongly agree that the time it takes to approve a records schedule is satisfactory. About half of all respondents (51.3%) affirmatively state that the length of time is unsatisfactory. In addition, this question received by far the largest number of "strongly disagree" responses from survey participants (21 or 18.3%). This almost equals one in every five respondents. About one in six respondents (16.5%) replied that they are neutral about this issue.
When responses are analyzed by a survey participant's overall degree of satisfaction, it is evident that timeliness of the approval process may be the single most important issue to the vast majority of dissatisfied FROs. Twenty out of twenty-four dissatisfied FROs disagree or strongly disagree that the time it takes to approve a records schedule is satisfactory (83.3%).
Respondents who categorized their overall satisfaction level as neutral or satisfied also express their highest level of concern in this area. Seventeen out of 26 neutral FROs (65.4%) and a third of satisfied FROs (22 out of 65 for 33.8%) either disagree or strongly disagree that the time it takes to approve a records schedule is satisfactory.
Analysis of satisfied survey participants' responses, however, show that other factors, such as adequacy of NARA's communication and the utility of NARA's guidance products, apparently do influence overall satisfaction levels for some FROs. A respondent may disagree or strongly disagree that the amount of time it takes to approve a schedule is satisfactory, but still express overall satisfaction with scheduling and appraisal services.
When asked to identify the most important thing NARA could do to improve scheduling and appraisal services, 35 respondents of the 83 who commented (this equals 30.4% of respondents who submitted comments) indicate that the schedule approval time should be shortened and included ideas on how to accomplish this. Twenty-two of these respondents (equaling 19% of respondents who submitted comments) identify it as the single most important thing that could be done to improve services. (The other 13 respondents included a comment about this issue among other recommendations.) Comments on this subject were received from respondents across the board. Respondents who are satisfied (16), neutral (nine), and dissatisfied (10) all submitted negative comments about the schedule approval time.
Samples of comments NARA received regarding ways to shorten the amount of time it takes to approve schedules include the following:
- Provide ways to fast-track certain jobs (e.g., those with no permanent records or legal retention requirements, common program records such as program planning and policy, and routine schedules);
- Provide ways to decrease the Federal Register time (by submitting notices more frequently; shortening the amount of time allotted for comment; and posting all jobs to the public as a way to reduce the 30-day wait to see if the job is requested);
- Allow staff to devote more time to scheduling (by hiring more staff or decreasing the amount of time staff must spend on special projects); and
- Mail signed schedules to agencies more quickly.
The next highest number of comments received (17) recommended changes in the scheduling method. Many of these recommendations, but not all, are proposals that might reduce the amount of time it takes to review and authorize a schedule. Samples of these comments include the following:
- Develop a template for the SF 115;
- Accept schedules electronically;
- Hold appraisal archivists accountable for the jobs they receive;
- Remove the policy of having to re-appraise records when the format changes (i.e. allow for media neutrality);
- Require NARA Stakeholder Units to conduct an on-site records appraisal if they recommend something as permanent;
- Reduce appraisal archivist staff turnover (respondent had four different appraisal archivists in three years);
- Supply the agency with a list of questions to ask the program office so the agency can provide the information to NARA up-front;
- Delegate authority for determining retention periods for temporary records to the originating agency;
- Encourage big bucket scheduling;
- Conduct on-line appraisals of records through e-mail conferences, etc.;
- Devise a scheduling methodology and techniques in support of Electronic Record Keeping Systems, in terms of organization of records information and taxonomy that cover GRS, Program Records and Information Systems;
- Allow for agency records management staff to work more closely with an appraisal archivist when drafting the schedule;
- Assign deadlines and request that NARA headquarters staff stay in the loop when delegating appraisal jobs to NARA field offices; and
- Send copies of the 115 to originator of the schedule in addition to the FRO.
Survey Question 5: NARA produces guidance on records scheduling policy and procedures in a timely fashion.
Survey Question 6: NARA guidance on records scheduling policy and procedure is comprehensive enough to meet my agency's scheduling needs.
Survey Question 7: It is easy to work with NARA guidance products containing records scheduling policy and procedures.
Survey Question 8: My agency receives consistent guidance from NARA staff regarding records scheduling policy and procedures.
The responses of FROs to the four questions about NARA guidance fell in the mid-range of ratings and were roughly equivalent, ranging from 60.8% to 64.3% of all respondents agreeing or strongly agreeing that NARA scheduling and appraisal guidance is produced in a timely fashion, comprehensive enough, easy to work with, and consistently interpreted by NARA staff. Questions 6-8 elicited the highest percentage of "neutral" responses from survey participants, ranging from 19.1% to 21.7%. In other words, about one in five survey respondents identify themselves as neutral on most NARA guidance products questions.
Satisfied and dissatisfied FROs differed widely in their responses to these questions. More than half of dissatisfied FROs (13 out of 24 for 54.2%) disagreed or strongly disagreed with Questions 5, 7, and 8. Question 6, regarding the comprehensive nature of NARA's guidance products, received a better rating by this group of FROs (11 out of 24 disagreed or strongly disagreed for 45.8%). On the other hand, no more than 6.3% (4 out of 63) of satisfied FROs disagreed or strongly disagreed with any of these four questions. This is a far cry from the 33.8% of satisfied FROs who disagreed with Question 1.
Respondents address a number of issues about guidance products in the comments section of this survey. Many comments focus on the GRS. Respondents request NARA to expand the GRS, apply big-bucket concepts to the GRS, and distribute the GRS on compact disk. Examples of other recommendations include:
- Provide examples/samples to accompany guidance products (e.g., of big bucket and media neutral schedules);
- Train archivists how to implement new initiatives so they can answer agency questions after BRIDG meetings;
- Provide guidance on changing technology; and
- Simplify guidance.
Survey Question 2: NARA staff keeps my agency informed about the progress of our records schedules throughout the approval process.
Survey Question 3: When my agency's staff contacts our NARA appraisal archivist for assistance, we receive a response that meets our needs.
Survey Question 4: My agency's staff has a good working relationship with our NARA appraisal archivist.
Survey respondents give the highest ratings to some aspects of the communication it receives from NARA. The most respondents overall agree or strongly agree they have a good working relationship with their appraisal archivist (Survey Question 4) and receive a response from their appraisal archivist that meets their needs (Survey Question 3). However, Survey Question 2, relating to the adequacy with which NARA keeps FROs informed about the progress of their records schedule through the appraisal process, received the second-lowest rating from survey respondents overall.
Survey Question 4 received the highest rating from FROs. The vast majority of respondents (93 out of 115 for 80.9%) agree or strongly agree that they have a good working relationship with their NARA appraisal archivist. Thirteen percent indicate that they are neutral on this subject. Only four respondents (3.5%) express concern about this working relationship.
Survey Question 3 received the second highest rating. In response to this question, 75.7% of respondents agree or strongly agree that they receive a response that meets their needs when they contact their appraisal archivist. A little over 10 percent (10.4%) are neutral to this question and about 11 percent (11.3%) disagree or strongly disagree.
When respondents were asked to explain the most important thing NARA could do to improve its scheduling and appraisal services, 11 of the 83 respondents who submitted comments (equaling 11.3% of respondents who submitted comments) made recommendations or negative comments regarding the issue of receiving a response that meets their needs. Of these respondents, one is satisfied overall, three are neutral, and seven are dissatisfied overall. About half (six) of the comments NARA received on this topic request that archivists respond promptly to FROs' phone calls and e-mails. Other recommendations include providing FROs with a telephone list of appraisal archivist contacts, requesting appraisal archivists to keep their voice-mail up-to-date to indicate when they are out of the office, and providing more Targeted Assistance.
Regarding Survey Question 2, 49.5% of respondents agree that NARA staff keeps their agency informed about the progress of their records schedules throughout the approval process. This is the second lowest rating among the eight survey questions. In addition, a quarter of respondents (26.1%) disagree or strongly disagree with this statement while 14.8% are neutral and 9.6% indicate that this statement is not applicable to them. Only Question 1 received a higher level of disagreement from survey participants.
There appears to be a direct correlation between a survey respondent's overall satisfaction level and being kept informed by NARA about the status of their schedules. The vast majority of satisfied FROS (72.3%) agree that they are kept informed with only a small fraction (7.7%) stating that they are not kept informed. The majority of dissatisfied FROs (58.3%), however, state that they are not kept informed of the status of their schedules. One in five dissatisfied FROs (20.8%) agree they are kept informed.
Almost one in ten respondents (8.7%) submitted comments relating an interest in being kept appraised of the status of their schedules. Comments also show this is an issue that appears to be a concern primarily to dissatisfied respondents. Of the respondents, one is satisfied overall, two are neutral, and seven are dissatisfied. Two other respondents indicate in their comments they would like to be kept better informed about who is their appraisal archivist.
NARA is committed to improving its scheduling and appraisal services to raise customer satisfaction levels. NARA will carry out the following actions in order to meet the needs FROs express in this survey:
Timeliness of Schedule Approvals:
Survey participants across the board express the highest level of concern with the amount of time it takes to approve a records schedule (see Survey Question 1). Overall, 25.2% of respondents agree or strongly agree that the time it takes to approve a records schedule is satisfactory. About half of all respondents (51.3%) state that the length of time is unsatisfactory.
- NARA conducted a business process reengineering project and is currently analyzing the data gathered to identify ways to streamline the scheduling and appraisal processes.
- By the end of Fiscal Year 2007, NARA will implement an electronic Request for Records Disposition Authority (SF 115) as part of the first phase of its Electronic Records Archive (ERA).
Communication on Schedule Approval Status:
About one-quarter of respondents (26.1%) disagree or strongly disagree that NARA staff keeps their agency informed about the progress of their records schedules throughout the approval process (see Survey Question 2). The vast majority of satisfied FROs (72.3%) agree that they are kept informed while the majority of dissatisfied FROS (58.3%) state that they are not kept informed.
- NARA appraisal archivists will begin sending a monthly schedules status report to agencies in August 2005.
Utility of Guidance Products:
The majority of dissatisfied Federal Records Officers (54.2%) disagree or strongly disagree that NARA guidance products are timely and easy to use and that they receive consistent guidance from NARA staff (Survey Questions 5, 7, and 8). In addition, 45.8% of this survey's respondents disagree or strongly disagree that NARA guidance products are comprehensive enough to meet agency needs (Survey Question 6). Respondents address a number of issues about guidance products in the comments section of this survey. Many of these comments focus on the General Records Schedule (GRS).
- NARA has issued two new GRSs during fiscal year 2005: GRS 26 for records of Temporary Commissions, Boards, Councils and Committees in September 2004, and GRS 27 for Records of the Chief Information Officer in May 2005.
- NARA will post contact information on its website for the NARA staff that will accept agency proposals for records that might be covered by a new GRS.
- NARA issued guidance in September 2004, expanding currently acceptable formats to enable the transfer of permanent web content records. This added a sixth format to the list of NARA's expanded acceptable formats for permanent electronic records.
- NARA provided agencies with detailed guidance about flexible scheduling and examples of big bucket schedule items in April 2005.
The results of this survey show that timeliness, adequacy of communication, and utility of guidance products all influence overall satisfaction levels of NARA's customers with its scheduling and appraisal services. The largest number of survey respondents agree that they have a good working relationship with their appraisal archivist and that they receive a response that meets their needs they contact NARA for assistance. The largest number of survey respondents disagree that the time it takes to approve a records schedule is satisfactory and that they are being kept informed about the status of their schedules. The responses of FROs to the four questions about NARA guidance fell in the mid-range of ratings and were roughly equivalent. In order to raise the overall satisfaction level of its customers from 56.6% satisfied or very satisfied and 22.6% dissatisfied or very dissatisfied, NARA will focus on improving its services in three areas: timeliness of approving a records schedule, informing agencies about the progress of their records schedules throughout the approval process, and the utility of its scheduling guidance products.