Cooperative Audit Resolution and Oversight Initiative (CAROI) Pilot Program (HTML)

(CAROI) Pilot Program

Results in Brief

September 23, 2014

Six Principles of CAROI

Improve Audit Resolution

Improve Communication

Foster Collaboration

Promote Trust

Develop Understanding

Enhance Performance

Summary

The Office of Inspector General (OIG) of the United States Equal Employment Opportunity Commission (EEOC) is conducting a pilot process called the Cooperative Audit Resolution and Oversight Initiative (CAROI) [pronounced kuh-roy]. The OIG’s goal for this pilot program is to use CAROI to improve resolution of recommendations contained in reports that require audit follow-up. Since the OIG began using CAROI in 2012, we achieved significant results, including improved communication with a key program office, and we identified an audit follow-up process glitch. In this document we present an overview of the CAROI concept, why and how the OIG implemented CAROI, and we discuss the benefits of our pilot program efforts.

CAROI Background

In 1995, the U.S. Department of Education (ED), Office of Inspector General, sought an improved method for closing unresolved audit findings. Reasons for seeking a new method included the failure to eliminate the causes of grant program deficiencies and the desire to avoid lengthy legal battles over federal auditor’s findings that questioned how recipients spend the grants. For example, one recipient of federal ED grants, the Commonwealth of Pennsylvania, had a deficiency backlog of 119 matters. Some of the backlog dated back seven years. After working with internal and external stakeholders, including state and local partners, ED developed CAROI. Using CAROI, the ED OIG resolved the Pennsylvania backlog within six months. CAROI is still in use by ED. Other examples of CAROI include ongoing efforts by the U.S. Environmental Protection Agency (EPA) and the Office of Management and Budget (OMB). In Massachusetts, CAROI is used not only to work with the federal government (e.g., ED and the National Science Foundation) in resolving audits, but also within the state government by the Department of Elementary and Secondary Education.

CAROI Fundamentals

CAROI seeks to improve dialogue, promote innovation in identifying solutions to problems, foster continuous improvement of the audit process, improve the efficiency and effectiveness of all oversight activities, and minimize bureaucratic and other inefficient methods. CAROI is well suited to the OMB’s December 8, 2009 Open Government Directive M-10-06 (Directive). Both the Directive and CAROI stress the importance of using innovative methods to increase collaboration. The Directive requires federal agencies to take specific actions to implement the principles of transparency, participation, and collaboration.

Challenges to successfully using CAROI for improving audit resolution include (1) individual and organizational resistance to change and (2) identifying those opportunities for making audit resolution a process built on cooperation, dialogue, and improvement of services.

The OIG’s Decision to Use CAROI

The OIG issues audit reports and other work products containing recommendations for the EEOC. Auditees are required to address these recommendations using a written corrective action plan (CAP) and other documentation, as appropriate, to demonstrate how they plan to address the recommendations. The EEOC’s Office of Chief Financial Officer (OCFO) is the Audit Follow-up Official (AFO), as stated in the relevant EEOC directive.

Over the past several years, we noted more than a few problems with audit resolution at the EEOC, including but not limited to:

• Long-standing recommendations which have not been addressed in a timely manner;
• A lack of clarity in the audit resolution process; and
• A lack of meaningful communication between the AFO and the auditee regarding audit resolution.

In 2011, the OIG considered piloting CAROI to determine its viability as a tool to improve audit follow-up at the EEOC. The OIG also decided to implement CAROI in a manner suited to the EEOC’s organization and culture, and in line with the OIG’s resource constraints. One of CAROI’s strengths is flexibility of implementation. With this flexibility and the OIG’s limited staff and resources, the OIG decided to pilot CAROI in a measured manner, beginning with a single product.

The OIG chose its Evaluation of the Management of the EEOC’s State and Local Programs (Project Number 2010-09-AEP) completed in March 2011. We chose this work product for two main reasons: (1) the managers of the audited entity, the Office of Field Programs (OFP), were already familiar with the OIG conducting work in their program areas and (2) the number and types of recommendations (moderately complex) would require ongoing and effective communication to resolve.

Our Approach to Implementing CAROI

The OIG’s initial approach to implementing CAROI was more of an exploratory examination of the conceptual underpinnings of the doctrine, rather than a formal test of its efficacy in real-world applications. The implementation efforts were less scientific and structured in our approach than outlined in the Association of Government Accountants’ CAROI Guide. Our approach was not intended as a fundamental disregard for the CAROI Guide as much as it was an attempt to determine how well we understood the six principles of CAROI, and whether the six principles would work for a small agency in nontraditional audit conditions. Second, we had concern that a more formal approach, coupled with a limited familiarity with CAROI could create resistance to its’ implementation.

OIG felt that a low-keyed approach would reduce the risk of using an inordinate amount of resources on an uncertain outcome. For example, we chose not to execute a CAROI agreement, establish formal teams, or seek any specific outcomes (i.e., no specific goals were set for the resolution of recommendations) other than to ascertain whether CAROI was applicable on a broader scale. Our overarching goal was to gain fundamental knowledge about applying CAROI and whether it could be an effective tool for the EEOC’s audit follow-up organization. Further, we sought to help the auditee improve its experience with the audit resolution process, increase their program efficiency and effectiveness, and improve their performance.

What We Learned

1.     CAROI valuable tool in identifying anomalies and systemic flaws in an organization’s audit follow-up and resolution process. CAROI can remedy such glitches, resulting in improved audit resolution program efficiency and effectiveness, and improvement in overall program operations.
2.    CAROI can reduce the potential for the auditee’s unwise investment of valuable resources in developing and implementing CAPs that are complex and perhaps misguided, or implementing CAPs designed to correct problems that do not exist or have been resolved.
3.     The CAROI paradigm is a preemptive and preventative audit resolution methodology. It is well suited to avert recurring instances of internal control failures, thereby improving the system of internal control and fostering greater trust and improved collaboration among and between the audit organization, the auditee, and the audit resolution function. The EEOC’s current approach to audit resolution relies heavily on the assumption that the auditee is knowledgeable about audit resolution.

Internal Stakeholder Involvement

In July 2012, the OIG met with the Office of Field Programs (OFP) management and managers of the State and Local Programs to discuss the OIG’s CAROI initiative. Prior to the meeting, the OIG provided program staff with information (e.g., information about CAPs) to prepare for a substantive discussion of the principles and benefits of sound CAPs. During the meeting, the principles of CAROI were presented and discussed, as well as the benefits of CAROI, and how we envisioned developing and implementing CAROI.

As a result of the meeting, we determined that assisting the auditee with procedural and formatting issues of its CAP would be the initial steps in implementing CAROI. Because of this assistance and the auditee’s efforts, which included a good deal of dialogue, the auditee was able to develop a well-structured CAP. It was then implemented, resulting in closure of all recommendations in less time than would have been the case without the use of CAROI.

At this point, the OIG conducted interviews with the Director of the OFP and management and staff of the State and Local Programs. The interviewees stated the following:

1.     Communication between the OFP and the OIG improved significantly. The OIG better understood the concerns of the OFP; and the OFP more fully understood the concerns of the OIG regarding improvements in program efficiency and effectiveness.
2.      The use of CAROI could result in improved program and activity results, due to a reduced focus on compliance and greater focus on results.
3.      The OIG should continue using CAROI because it may result in improved communication and enhanced program effectiveness and efficiency.
4.      Although implementing the CAROI pilot program took significant time, the investment of time was worthwhile, due to the benefits detailed above.

Additional Benefits from the CAROI Pilot

Because of the CAROI pilot program, the OIG identified an audit resolution process issue (the delayed review of the CAP by the Audit Follow-up Official (AFO) involving the Evaluation of the Management of the EEOC’s State and Local Programs). This discovery was the direct result of the informal and flexible nature of CAROI (and a quality control check on the OIG’s semiannual report revealing a difference between the OCFO’s and the OIG’s understanding of the resolution of some recommendations).

The OIG then entered into discussions with the OFP and the OCFO to identify the cause of the delayed review of the CAP. Together, these groups identified the cause and, more importantly, confirmed the need to clarify the audit follow-up process and continue to communicate throughout the process. Additional OCFO and the OIG staff meetings resulted in clarifications of the audit follow-up process and agreement on the need for a memorandum of understanding to institutionalize the audit follow-up process.

Prior to the end of our last reporting period (October 2013–March 2014), the OIG evaluation staff and AFO staff discussed a CAP for another evaluation report, resulting in improved AFO assessment of the CAP. In February and March of 2014, we met with the OCFO audit resolution officials and discussed the CAP for the March 2013 OIG report EEOC Performance Measures. The discussion, which centered on the structure of the CAP, led to clear and accelerated communication between the AFO and the auditee. In turn, that resulted in clarification of the auditee’s position on addressing several of the recommendations. This allowed the AFO to more readily and objectively decide whether to close recommendations. We are pleased with the additional transparency this has brought to the closeout process. In addition, the AFO reports that CAROI has improved its interactions with the OIG, resulting in greater opportunities to discuss the intent of recommendations, and whether proposed corrective actions adequately address those recommendations.

We believe that consistent and more widespread use of CAROI will result in timelier disposition of recommendations. This should lead to better products with more practical recommendations.

Next Steps

The OIG plans to use CAROI selectively on future evaluations and performance audits, and to continue to provide results of the pilot program to, and solicit feedback from, EEOC management and staff involved in audit follow-up. We also believe that CAROI may be useful for the EEOC when resolving audit and related findings with oversight organizations other than the OIG, such as the Government Accountability Office (GAO) and the U.S. Office of Personnel Management. It should be noted that CAROI can be an effective resource for resolving audit and related findings regardless of whether the audits are subject to the audit follow-up process.