TABLE OF CONTENTS

EXECUTIVE SUMMARY

A. OVERVIEW AND PURPOSE

B. METHODOLOGY

B1 Reviewing the Literature

B2. Interviewing EEOC Leadership

B3. Synthesizing Information

C. FINDINGS

C1. Improving the Consistency of Intake Procedures

C2. Determining the Significance of a Charge

C3. Addressing Systemic Discrimination

C4. Improving Data Collection

D. SUMMARY OF RECOMMENDATIONS

E. AGENCY COMMENTS AND OUR EVALUATION

APPENDIX A: REFERENCES

APPENDIX B: PRIORITY CHARGE HANDLING PROCESS (PCHP)

APPENDIX C: FY 2012-2016 STRATEGIC PLAN

APPENDIX D: EEOC LEADERSHIP AND STAFF INTERVIEW GUIDE

APPENDIX E: CONVERSATION WITH DISTRICT DIRECTORS PROTOCOL

APPENDIX F: VALIDITY OF THE SELECTED STUDIES

APPENDIX G: COMMENTS FROM THE OFFICE OF THE CHAIR

APPENDIX H: COMMENTS FROM THE OFFICE OF FIELD PROGRAMS

APPENDIX I: COMMENTS FROM THE OFFICE OF RESEARCH, INFORMATION, AND PLANNING

EXECUTIVE SUMMARY

The Equal Employment Opportunity Commission (EEOC) Office of the Inspector General (OIG) contracted with IMPAQ International, LLC in September 2012 to conduct a review of evaluations of EEOC's private sector enforcement program. This was the first review of evaluations contracted by the EEOC's OIG. The purpose of the review is to provide EEOC leadership and stakeholders with recommendations to enhance the efficiency and effectiveness of EEOC private sector enforcement activities including mediation.⁽¹⁾ A thorough analysis based on research about new and useful approaches to enforcement will help inform the Commission and its workforce about EEOC's strengths and concerns. Study recommendations are geared toward aiding the EEOC in its efforts to improve the effectiveness and efficiency of its private sector enforcement activities.

Overview and Purpose. The review focused on synthesizing information from evaluations examining EEOC's key charge processing activities, as well as input obtained through interviews with EEOC leadership. The following research questions guided the review:

• What factors are associated with the efficiency with which the EEOC carries out private charge processing activities?
• What factors are associated with the effectiveness with which the EEOC carries out private charge processing activities?
• What are the key information gaps related to developing strategies to more efficiently and effectively carry out private sector charge processing activities?

EEOC's litigation activities were not part of the scope of the review.
Methodology. This project had three overarching tasks. The first task to review the recent literature examining private sector enforcement activities of the EEOC was executed during the period of September - November 2012. Numerous data sources including online law literature (e.g., HeinOnline) and research databases (e.g., EBSCOHost, Google Scholar), as well as online library resources at the University of Maryland and The Johns Hopkins University were used to identify and retrieve relevant studies. The literature search included peer-reviewed journal articles, government reports, books, integrated reviews of relevant literature, meta-analyses, and grey literature (e.g., unpublished reports and conference proceedings). To avoid omissions of key studies, the project team used a snowballing technique to identify publications that were not uncovered during the initial search. This approach entailed reviewing the bibliographies and reference lists of the publications identified during the initial search in order to identify other relevant studies. The Contracting Officer's Technical Representative (COTR) also assisted in identifying and obtaining relevant documents. Studies as current as 2013 were identified. The period of the last 10 years (since 2003) was considered the most relevant, although older studies were included if pertinent. Studies obtained through the literature search were assessed for relevance to the project and validity. Relevant EEOC documents were reviewed, as well.

The second task was interviewing EEOC leadership to obtain their insights regarding the effectiveness and efficiency of private charge processing activities. In collaboration with the COTR, the project team scheduled telephone interviews with key individuals. The following individuals from the EEOC were interviewed:

• Chief Operating Officer
• Director, Office of Research, Information and Planning
• Director, Office of Field Programs
• Operations and Policy Specialist, Office of Field Programs
• Attorney Advisor, Office of Field Programs
• Three District Directors

Interview protocols developed and followed to gather information in a systematic manner are provided as appendices. All interviews were conducted in either October 2012 or February 2013.
The third task was to synthesize this information to identify recommendations to aid the EEOC in improving the effectiveness and efficiency of its private sector enforcement activities. This task was carried out throughout the course of the project from October 2012 - February 2013. The information synthesis identified four areas related to the Priority Charge Handling Process (PCHP) in which recommendations to improve effectiveness and efficiency could be made. The areas are (1) improving the consistency of intake procedures, (2) determining the significance of a charge, (3) addressing systematic discrimination, and (4) improving data collection.

The COTR provided a draft of this report to the EEOC for their review and comment. The Office of the Chair (OC), Office of Field Programs (OFP), and Office of Research, Information, and Planning (ORIP) provided written comments, which have reprinted in appendices G, H and I, respectively. Technical elements of these comments have been incorporated as appropriate.

Recommendations. Based on the review of the literature and the interviews conducted in this study, the following recommendations are made to enhance the effectiveness and efficiency of EEOC's private sector charge processing:

1. The EEOC should further standardize intake procedures across field offices. Consistent with ongoing efforts to improve the uniformity of intake materials and other efforts to establish greater communication with and between field offices, best practices related to the intake interview should be identified and widely adopted with accompanying training as needed. For example, field staff in some offices have adopted the practice of being "frontal" with charging parties. In this approach, field staff gather during the intake interview as much information as possible while simultaneously clearly explaining the law and the relief to be expected to the charging party. The intent of this approach is to have the decision to file a charge be made with clear expectations and to avoid instances in which the charging party files a charge "just to file a charge." The Office of Field Programs is working toward standardizing intake procedures across field offices. Oversight by EEOC leadership to ensure that progress continues in this area is encouraged.
2. The EEOC should document criteria for determining Category C charges. To enhance efficiency and effectiveness and promote the delivery of excellent and consistent service as sought in the Strategic Plan, the EEOC should continue to pursue ways to improve its triaging of charges, especially with regard to identifying Category C charges. The criteria used for assessing the significance of a charge vary across field staff and case characteristics, but it is challenging to standardize criteria. Accordingly, the EEOC should consider new approaches to efficiently determining Category C charges to reduce the inventory of pending charges and enable field staff to devote limited resources to effectively identifying and addressing potentially significant charges (i.e., Category A and B charges). The PCHP guidelines, the EEOC FY 2012-2016 Strategic Plan, and the Strategic Enforcement Plan define how to categorize and thus pursue a charge. Nevertheless, detailed guidelines are needed to aid consistency across field staff in determining whether a charge should be classified as Category C. Criteria for determining whether a charge is Category C should be distributed widely across field offices with appropriate training for field staff. Field office managers should be instructed to oversee adherence to the criteria. Additionally, to further improve consistency, a standardized set of parameters and keywords based on these criteria should be incorporated into the EEOC Integrated Mission System (IMS) database to allow field staff during their case research to refer to determinations of similar cases across field offices.
3. The EEOC should continue efforts to develop a national approach for addressing systemic discrimination. The EEOC has made progress in recent years addressing system discrimination. The Commission has set a target that by FY 2016, 22-24 percent of the cases on its active litigation docket will constitute systemic cases. This is an increase from the 20 percent baseline of FY 2012. To attain the FY 2016 goal, we encourage the EEOC to remain committed to identifying systemic discrimination through a strategic, nationwide, and coordinated approach aided by newly adopted and future technological capabilities, excellent guidance from subject experts, and continued oversight by EEOC leadership.
4. The EEOC should continue to review the range of information obtained during intake interviews and how it is stored in the IMS. Along with further standardizing intake procedures, the EEOC should examine the information that is currently obtained from charging parties and assess whether every element of information is essential and whether additional information is needed for a given charge or for addressing systemic discrimination. In addition, assessments should be made regarding how to more effectively store intake information in the IMS by identifying and defining standardized phrases, clauses, parameters, and keywords to enable field staff to input data more consistently and investigate charges more effectively and efficiently.
5. The EEOC should investigate the merits of expanding the information it obtains related to hiring and terminations. The EEOC should investigate whether it would be advantageous to its mission to obtain more information. Currently, much of the data collected on hiring and termination is determined by law. The EEOC should consider both obtaining additional information from the employers, employees, and applicants currently providing information, and expanding the groups of employers and employees required to provide the EEOC with information. This is a key information gap. Should the EEOC determine that additional information would enhance its effectiveness and efficiency in addressing employment discrimination, it should assess what changes to procedures, rules, and laws would be necessary.

OVERVIEW AND PURPOSE

The Equal Employment Opportunity Commission (EEOC) Office of the Inspector General (OIG) contracted with IMPAQ International, LLC in September 2012 to conduct a review of evaluations of EEOC's private sector enforcement program. This was the first review of evaluations contracted by the EEOC's OIG. The purpose of the review is to provide EEOC leadership and stakeholders with recommendations to enhance the efficiency and effectiveness of EEOC private sector enforcement activities including mediation.⁽²⁾ A thorough analysis based on research about new and useful approaches to enforcement will help inform the Commission and its workforce about EEOC's strengths and concerns. Study recommendations are geared toward aiding the EEOC in its efforts to improve the effectiveness and efficiency of its private sector enforcement activities.

The review focused on synthesizing information from evaluations examining EEOC's key charge processing activities, as well as input obtained through interviews with EEOC leadership. The following research questions guided the review:

• What factors are associated with the efficiency with which the EEOC carries out private charge processing activities?
• What factors are associated with the effectiveness with which the EEOC carries out private charge processing activities?
• What are the key information gaps related to developing strategies to more efficiently and effectively carry out private sector charge processing activities?

EEOC's litigation activities were not part of the scope of the review.

The EEOC is the key Federal agency responsible for enforcing Federal laws that prohibit employment discrimination based on a person's age (40 or older), race, sex (including pregnancy), color, religion, national origin, disability, or genetic information. These laws cover both small and large employers, including those in the private sector, the public sector, and nonprofits, and pertain to all aspects of employment including hiring, training, wages, benefits, harassment, promotion, and termination. The EEOC has the authority to investigate charges of systemic or individual discrimination against employers who are covered by these laws.⁽³⁾ The EEOC's private sector charge processing system registers discrimination charges based on a variety of types of discrimination, including disability, HIV/AIDS status, cancer status, sexual harassment, and race.⁽⁴⁾ To enhance efficiency, the EEOC adopted the Priority Charge Handling Procedure (PCHP) in April 1995, which serves as its current system of case triage.⁽⁵⁾ For a variety of reasons, the inventory of pending charges at year-end nearly tripled from FY 2002 (29,041)⁽⁶⁾ to FY 2010 (86,338).⁽⁷⁾ More recently, the EEOC has made progress in reducing the inventory. In the last two years, the inventory was reduced to 78,136 in FY 2011⁽⁸⁾ and 70,312 in FY 2012.⁽⁹⁾ To put the reduction in the inventory in context, in FY 2012 the EEOC received 99,412 private sector workplace discrimination charges and resolved 111,139 charges.⁽¹⁰⁾

The Commission's FY 2012-2016 Strategic Plan⁽¹¹⁾ emphasizes its commitment to serve the nation more efficiently. The Strategic Plan is based upon three values-commitment to justice, accountability, and integrity-and departs from previous strategic plans by focusing less on measuring output and more on achieving long-term goals. As such, the Strategic Plan requires the Commission to be proactive and prioritizes a coordinated approach to law enforcement.

The Strategic Plan outlines three objectives, each with its own set of outcome goals:

1. Combat employment discrimination through strategic law enforcement, with outcome goals of (1) having a broad impact on reducing employment discrimination at the national and local levels, and (2) remedying discriminatory practices and securing meaningful relief for victims of discrimination.
2. Prevent employment discrimination through education and outreach, with outcome goals of (1) better informing the public about how to exercise their right to employment free of discrimination, and (2) informing employers, unions, and employment agencies (covered entities) about how to better address and resolve Equal Employment Opportunity (EEO) issues, thereby creating more inclusive workplaces.
3. Deliver excellent and consistent service through a skilled and diverse workforce and effective systems,with an outcome goal of timely, informative, and high quality interactions with the public.

The recommendations in this review will assist the EEOC in its efforts to meet the objectives and goals of the FY 2012-2016 Strategic Plan. Section B of this report describes the methodology employed in the review. Section C presents the findings of the synthesis, grouping the evidence into four broad categories:

• Improving the consistency of intake procedures
• Determining the significance of a charge
• Addressing systemic discrimination
• Improving data collection.

Section D summarizes the recommendations for ease of reference.

METHODOLOGY

This project had three overarching tasks. The first task was to review the recent literature examining private sector enforcement activities of the EEOC was executed during the period of September - November 2012. The second was interviewing EEOC leadership to obtain their insights regarding the effectiveness and efficiency of these activities. All interviews were conducted in either October 2012 or February 2013. The third task was to synthesize this information to identify recommendations for aiding the EEOC in improving the effectiveness and efficiency of its private sector enforcement activities. This task was carried out from October 2012 - February 2013.

B1. Reviewing the Literature

The IMPAQ team conducted a literature search examining the private sector charge processing activities of the EEOC. Numerous data sources were used to identify and retrieve relevant studies. Sources included online law literature (e.g., HeinOnline) and research databases (e.g., EBSCOHost, Google Scholar), as well as online library resources at the University of Maryland and The Johns Hopkins University. The systematic search began with four primary keywords: (1) EEOC, (2) private sector employment discrimination, (3) EEOC and private sector charges, and (4) private charge handling process. The literature search included peer-reviewed journal articles, government reports, books, integrated reviews of the literature, meta-analyses, and grey literature (e.g., unpublished reports and conference proceedings). The search focused on studies published after 2003 to ensure that the studies examined the period after the EEOC adopted the PCHP.

Although the literature search focused on the private sector charge processing activities of the EEOC, the IMPAQ team also reviewed publications describing the inception, organization, functioning, and responsibilities of the EEOC to understand the challenges faced in addressing charges. To avoid omissions of key studies, the project team used a snowballing technique to identify publications that were not uncovered during the initial search. This approach entailed reviewing the bibliographies and reference lists of the publications identified during the initial search in order to identify other relevant studies. In addition, if the project team was unable to retrieve a relevant document online, a member of the team contacted the author directly and requested a copy of the publication. The COTR also assisted the project staff in identifying and obtaining relevant documents. Studies as current as 2013 were identified. The period of the last 10 years (since 2003) was considered the most relevant, although older studies were included if pertinent.

Studies obtained through the literature search were assessed for relevance to the project and overall validity as detailed in Appendix F. The IMPAQ team facilitated the assessment by extracting key information from each study and then developing a summary of each study. Included in each summary was information about authorship, funding, time period, and geographical area addressed. The project team also reviewed the methodology, outcomes, findings, and limitations of each study and added this information to the summary. The assessment of the relevance of a study focused on the research question(s) addressed by the study and the time period examined. Studies deemed relevant addressed a variety of research questions and sub-questions related to the organization, functioning, responsibilities, types of charges, and activities performed by the EEOC related to private sector charge processing. The period of the last 10 years (since 2003) was considered the most relevant, although older studies were included if pertinent. Appendix F describes in detail how studies were assessed for validity. Key factors for assessing validity included the reputation of the journal/research outlet, study methodology, and the study population/geography. Relevant EEOC documents were reviewed, as well.

B2. Interviewing EEOC Leadership

The IMPAQ team sought further insights into private sector enforcement activities from EEOC leadership and staff. In collaboration with the COTR, the project team scheduled telephone interviews with key individuals. The interviews were conducted in October 2012 and February 2013. The following individuals from the EEOC were interviewed:

• Chief Operating Officer
• Director, Office of Research, Information and Planning
• Director, Office of Field Programs
• Operations and Policy Specialist, Office of Field Programs
• Attorney Advisor, Office of Field Programs
• Three District Directors

Interview protocols were developed and followed in order to gather information in a systematic manner. The interview protocols developed for these interviews are presented in Appendices D and E. The semi-structured interviews began with an introduction and an explanation of the nature of the project and the interview. At that time, the interviewer requested permission to record the interview to facilitate note taking and ensure accuracy. The interviewee was also informed that all responses would remain confidential and that the interviewee would not be identified in the report. The interview questions were developed to allow interviewees to provide their views, concerns, and recommendations regarding improving the effectiveness and efficiency of EEOC private sector enforcement activities.

B3. Synthesizing Information

IMPAQ utilized content analysis to analyze the data from the literature review and the interviews. Project staff applied an inductive data coding strategy,⁽¹²⁾ in which the data drive the identification of themes or coding categories. For example, when analyzing data from the interviews, the project analysts initially categorized the data at the question level. Based on the responses to the questions, secondary categories were identified, creating a hierarchical structure. As more qualitative data were reviewed, new categories were created. Finally, the project analysts adjusted the categories to ensure that they were inclusive and exhaustive. This process is shown in Exhibit 1.

The IMPAQ project analysts summarized the findings within each category and then across categories in order to provide a clear and succinct description of the findings. The literature review and the interviews revealed both reinforcing and conflicting viewpoints. In instances in which viewpoints conflicted, the project analysts further examined the evidence supporting each viewpoint and identified any gaps in current knowledge.

Exhibit 1. Synthesis Methodology

Initial read- through of text data	Identify specific segments of information	Label the segments of information to create categories	Reduce overlap and redundancy among the categories	Create a framework with the most important categories
Many pages of text	Many segments of text	Initial list of categories	Consolidated list of categories	Categories that are relevant to the research questions

FINDINGS

The information synthesis identified four areas related to the PCHP in which recommendations to improve effectiveness and efficiency could be made: (1) improving the consistency of intake procedures, (2) identifying a significant charge, (3) addressing systemic discrimination, and (4) improving data collection. Per the nature of this review, recommendations at times build on previous recommendations.

C1. Improving the Consistency of Intake Procedures

A principal objective of the EEOC FY 2012-2016 Strategic Plan is to deliver excellent and consistent service that results in timely, informative, and high quality interactions with the public.⁽¹³⁾ Providing consistent service starts with providing and obtaining consistent information during the initial intake interview. The intake interview is the first step of determining the nature of a discrimination charge and how it will be pursued. It has been found that when charging parties do not receive consistent information at intake regarding their rights, opportunities, and obligations, it can lead to some charging parties dropping charges that would otherwise be found meritorious.⁽¹⁴⁾

Intake information offered to customers and intake procedures vary across field offices.⁽¹⁵⁾, ⁽¹⁶⁾ Evidence suggests that improvements have been made in recent years in addressing consistency at intake across field offices.⁽¹⁷⁾ In particular, the Office of Field Programs has adopted uniform intake questionnaires, brochures and checklists for field offices. Training related to intake, interviewing and the PCHP has been provided to staff, as well, to further promote consistency at intake.⁽¹⁸⁾

Nevertheless, achieving consistency across field offices is challenging.⁽¹⁹⁾ While all field offices are equipped with accredited staff trained to file and pursue charges, field offices differ in terms of the population served, available resources, and volume of charges filed.⁽²⁰⁾ As a result, field offices differ in how they strike a balance between addressing charges and providing intake services to charging parties. The most notable difference relates to the amount of time spent conducting the intake interview with the charging party. Interviewees agree that determining the significance of a charge is "more art than science and requires excellent interpersonal skills." ⁽²¹⁾ Variation in the depth of the information obtained at intake likely reduces consistency in charge processing and priority classification across field offices⁽²²⁾ and may be a contributing factor in the systematic differences in charge determinations by charging party characteristics found in some studies. ⁽²³⁾ Additionally, data entered into the IMS system will reflect any intake differences across field offices and affect the reliability of IMS data for analysis.

Consistent with ongoing efforts to improve the uniformity of intake materials and other efforts to establish greater communication with and between field offices,⁽²⁴⁾ best practices related to the intake interview should be identified and widely adopted with accompanying training as needed. For example, field staff in some offices have adopted the practice of being "frontal" with charging parties.⁽²⁵⁾ In this approach, field staff gather during the intake interview as much information as possible while simultaneously clearly explaining the law and the relief to be expected to the charging party. The intent of this approach is to have the decision to file a charge be made with clear expectations and to avoid instances in which the charging party files a charge "just to file a charge."⁽²⁶⁾ The Office of Field Programs is working toward standardizing intake procedures across field offices. Oversight by EEOC leadership to ensure that progress continues in this area is encouraged.

C2. Determining the Significance of a Charge

In order to be both effective and efficient, the EEOC instructs field office staff to focus on charges of discrimination deemed significant.⁽²⁷⁾,⁽²⁸⁾,⁽²⁹⁾ Under the PCHP, each incoming charge receives a priority assessment and is assigned a priority status (Category A, B, or C). Priority status assignment is based on information provided by the charging party (the alleged victim of discrimination) during the initial intake interview and on the field staff's perception of the credibility of the charging party. Category A indicates that the charge falls within the national or local enforcement plan, or that further investigation is likely to lead to a finding of cause. Category B indicates that additional evidence will be required to determine whether further investigation will reveal discrimination. Category C charges are those in which the office has sufficient information to conclude that it is not likely that further investigation will result in a cause finding.⁽³⁰⁾ Establishing a precise definition of what constitutes a significant charge is difficult, especially in relationship to the priority status of a charge.⁽³¹⁾,⁽³²⁾ Some criteria for categorizing a charge as significant are straightforward. For example, if the discrimination charge refers to egregious harm caused to an individual, then it is categorized as significant. Charges of systemic discrimination and workplace discrimination charges which are endemic in a particular industry also constitute significant charges.⁽³³⁾,⁽³⁴⁾ In such cases, current intake procedures that include the uniform collection of background information through standardized intake questionnaires and "layers of review to determine the charge category"⁽³⁵⁾ are adequate. Determining whether a charge is significant based on its potential to develop the law or influence policy making; however, is not as straightforward as it is unclear how to determine such potential.⁽³⁶⁾ Moreover, evidence suggests that prioritizing a discrimination charge depends on the presentation of the case by the alleged victim. For example, cases presented by a legal representative are often perceived as being more credible and more likely to be assessed as significant.⁽³⁷⁾,⁽³⁸⁾,⁽³⁹⁾,⁽⁴⁰⁾ On the other hand, field staff experience suggests that legal representatives also may guard information or provide information in "piecemeal" during intake to prevent the use of such information during a subsequent investigation.⁽⁴¹⁾ Variation in charge assessment criteria across subpopulations (e.g., individuals with HIV/AIDS, cancer) and across investigating staff has been found in studies, with investigators burdened with a high flow of charges being more resistant to deeming a charge significant.⁽⁴²⁾,⁽⁴³⁾ Compounding the problem is the lack of consistency in intake procedures across field offices.

To enhance efficiency and effectiveness and promote the delivery of excellent and consistent service as sought in the Strategic Plan, the EEOC should continue to pursue ways to improve its triaging of charges, especially with regard to identifying Category C charges. The criteria used for assessing the significance of a charge vary across field staff and case characteristics as described above, but it is challenging to standardize criteria.⁽⁴⁴⁾ Accordingly, the EEOC should consider new approaches to efficiently determining Category C charges to reduce the inventory of pending charges and enable field staff to devote limited resources to effectively identifying and addressing potentially significant charges (i.e., Category A and B charges).⁽⁴⁵⁾ The PCHP guidelines, the EEOC FY 2012-2016 Strategic Plan, and the Strategic Enforcement Plan define how to categorize and thus pursue a charge. Field offices provide training to field staff on the application of PCHP guidelines to improve consistency in charge categorization.⁽⁴⁶⁾ Nevertheless, detailed guidelines are needed to aid consistency across field staff in determining whether a charge should be classified as Category C.⁽⁴⁷⁾ Criteria for determining whether a charge is Category C should be distributed widely across field offices with appropriate training for field staff. Field office managers should be instructed to oversee adherence to the criteria. Additionally, to further improve consistency, a standardized set of parameters and keywords based on these criteria should be incorporated into the EEOC Integrated Mission System (IMS) database to allow field staff during their case research to refer to determinations of similar charges in other field offices.

C3. Addressing Systemic Discrimination

The EEOC defines systemic discrimination as "a pattern or practice, policy, or class case where the alleged discrimination has a broad impact on an industry, profession, company or geographic area."⁽⁴⁸⁾ Examples of systemic discrimination include not hiring qualified women in traditionally male-dominated occupations, and illegal pre-employment inquiries about disability status. An integral part of the Commission's mission is to identify, investigate, and act upon cases of systemic discrimination.⁽⁴⁹⁾ The Commission's FY 2012-2016 Strategic Plan⁽⁵⁰⁾ and the Strategic Enforcement Plan 2013-2016 (SEP)⁽⁵¹⁾ document EEOC's commitment to pursue systemic investigations.
Evidence indicates that although the EEOC successfully investigates, conciliates, and litigates numerous systemic cases, it does not identify systemic discrimination in a consistent and coordinated manner.⁽⁵²⁾ This is not a new concern as reflected in the report of the 2006 Systemic Task Force.⁽⁵³⁾ Task force recommendations such as requiring District Offices to develop systemic discrimination enforcement plans have been adopted. Through the SEP, the EEOC reaffirms the approach and principles of the Systemic Task Force - that systemic enforcement must be strategic, nationwide, coordinated and adequately resourced.⁽⁵⁴⁾ The Commission is extending its partnerships with employer groups, state and Federal agencies, advocacy groups, the plaintiffs' bar and other organizations to identify and address discriminatory practices. These actions will be supported by enhanced outreach efforts to inform and educate employers to encourage the prevention of systemic discrimination.⁽⁵⁵⁾
Currently, the Agency's internal incentives and resources are better aligned to address systemic cases than they have been in the past.⁽⁵⁶⁾ Specific examples include training opportunities with appropriate support materials and technical tools and greater coordination and collaboration between field offices facilitated by Headquarters personnel. Other examples include the development of an internal EEOC systemic website and the Systemic Watch List Tool which uses IMS data to network EEOC offices investigating systemic cases by notifying investigators and attorneys when a related charge or inquiry is received or when existing charges meet specified systemic criteria.⁽⁵⁷⁾ Also, many field offices are having more senior investigators guide other investigators and social scientists and subject matter experts on board to help with systemic cases.⁽⁵⁸⁾

The EEOC has made progress in recent years addressing systemic discrimination. The Commission has set a target that by FY 2016, 22-24 percent of the cases on its active litigation docket will constitute systemic cases.⁽⁵⁹⁾ This is an increase from the 20 percent baseline of FY 2012. To attain the FY 2016 goal, we encourage the EEOC to remain committed to identifying systemic discrimination through a strategic, nationwide, and coordinated approach aided by newly adopted and future technological capabilities, excellent guidance from subject experts, and continued oversight by EEOC leadership.

C4. Improving Data Collection

The EEOC receives charges on a variety of topics. Information gathered at intake and throughout the investigation is recorded in the EEOC's IMS database.⁽⁶⁰⁾ The IMS database contains information on charges that have been processed and investigated by the EEOC, as well as information obtained from employers. Evidence suggests, however, that resource constrained field offices may not always update the IMS database in a timely manner as details of a case change (e.g., a case reassessment leads to a change in the initial charge categorization).⁽⁶¹⁾ Moreover, as noted above, since uniformity in intake procedures across field offices is a challenge, IMS data will reflect any variation in intake across field offices. Accordingly, the reliability and completeness of IMS data for analysis could be a concern. Indications are that the EEOC is addressing data infrastructure issues such as increasing transmission capacity (bandwidth) to better facilitate electronically maintaining administrative case documents.⁽⁶²⁾ Such electronic documentation will greatly enhance efficiency in addressing charges and the value of the IMS to field staff. Additionally, the IMS now includes new codes to identify, for example, the type of relief provided to charging parties and to track lesbian, gay, bisexual, and transgender (LGBT) cases.⁽⁶³⁾

Some information that might enhance the effectiveness and efficiency of the EEOC in areas such as determining the significance of a charge is not collected. For example, while the IMS compiles data about charges filed, information about the charging party's charge history for other charges is collected and stored for only specific charging parties and respondents.⁽⁶⁴⁾ More broadly, the information collected by the EEOC does not adequately reflect the full workforce.⁽⁶⁵⁾ For example, the EEOC does not capture information from small private employers covered under Title VII such as those with fewer than 100 employees that are not affiliated or owned by another company.⁽⁶⁶⁾ Also, EEOC forms require reporting the number of employees by race, ethnicity, and gender in each job category, but other key information such as salaries by race and gender is not collected.⁽⁶⁷⁾ To identify early signs of employment discrimination, it would be useful to capture differences, if any, between the demographic distribution of the applicant pool and that of new hires, for each job position in each industry. However, providing such applicant data is currently largely voluntary. Also lacking is information on employee terminations.⁽⁶⁸⁾ Moreover, temporary hiring agencies are not required to collect and provide EEO-1 data. So, it is difficult to track any employment discrimination against those who use the services of these agencies.⁽⁶⁹⁾

Along with further standardizing intake procedures, the EEOC should examine the information that is currently obtained from charging parties and assess whether every element of information is essential and whether additional information is needed for a given charge or for addressing systemic discrimination. In addition, assessments should be made regarding how to more effectively store intake information in the IMS by identifying and defining standardized phrases, clauses, parameters, and keywords to enable field staff to input data more consistently and investigate charges more effectively and efficiently.

The EEOC should investigate whether it would be advantageous to its mission to obtain more information. Currently, much of the data collected on hiring and termination is determined by law. The EEOC should consider both obtaining additional information from the employers, employees, and applicants currently providing information, and expanding the groups of employers and employees required to provide the EEOC with information. Should the EEOC determine that additional information would enhance its effectiveness and efficiency in addressing employment discrimination, it should assess what changes to procedures, rules, and laws would be necessary.

D. SUMMARY OF RECOMMENDATIONS

Based on the review of the literature and the interviews conducted in this study, the following recommendations are made to enhance the effectiveness and efficiency of the EEOC's private sector charge processing:

1. The EEOC should further standardize intake procedures across field offices.
2. The EEOC should document criteria for determining Category C charges.
3. The EEOC should continue efforts to develop a national approach for addressing systemic discrimination.
4. The EEOC should continue to review the range of information obtained during intake interviews and how it is stored and updated in the IMS.
5. The EEOC should investigate the merits of expanding the information it obtains related to hiring and terminations.

E. AGENCY COMMENTS AND IMPAQ'S RESPONSE

The COTR provided a draft of this report to the EEOC for their review and comment. The Office of the Chair (OC), Office of Field Programs (OFP), and Office of Research, Information, and Planning (ORIP) provided written comments, which have reprinted in appendices G, H and I, respectively. Technical elements of these comments have been incorporated as appropriate.

OC indicated general agreement with the report recommendations. It noted that current agency strategic planning and implementation efforts are consistent with the purpose and recommendations of this report. OFP generally concurred with the report recommendations indicating that efforts to implement the Strategic Plan for FY 2012 - 2016 and the Strategic Enforcement Plan are consistent with observations in the draft report. Recommendations by OFP to acknowledge recent efforts to reduce the inventory of private sector charges and coordinate systemic discrimination activities have been incorporated in the report. ORIP disagreed with the methodology of the evaluation. As described in the report, the study employed a rigorous methodology to identify and select the most relevant research, then synthesized that information along with information drawn from several stakeholder interviews into the recommendations provided in this report. The approach suggested by ORIP in which the analysis would involve the extraction, evaluation and regeneration of data to produce more quantitative results was beyond the scope and resources of this study. Greater specificity has been incorporated as appropriate.

APPENDIX A: REFERENCES

Burris, S., Moss, K., Ullmann, M.D., Johnsen, M. (2000). Disputing Under The Americans with Disabilities Act: Empirical Answers, and Some Questions. Temple Political and Civil Rights Law Review, 9(2), 237-252.

CFI Group and Federal Consulting Group. (2008). Equal Employment Opportunity Commission Customer Satisfaction and Knowledge of Law Study. Washington, DC: Equal Employment Opportunity Commission.

Conyers, L., Boomer, K.B., McMahon, B.T. (2005a). Workplace Discrimination and HIV/AIDS: The National EEOC ADA Research Project.Work, 25(1), 37-48.

Conyers, L., Unger, D., Rumrill, Jr., P.D. (2005b). A Comparison of Equal Employment Opportunity Commission Case Resolution Patterns of People with HIV/AIDS and Other Disabilities. Journal of Vocational Rehabilitation, 22(3), 171-178.

Development Services Group. (2006). Evaluation of Intake and End-of-Fiscal-Year Closures of the Equal Employment Opportunity Commission Private Sector Charge Process. Bethesda, MD: Development Services Group.

EEOC. (2002). Combined Annual Reports Fiscal Years 1999-2001 (October 1, 1998 through September 30, 2001). Washington, DC: U.S. Equal Employment Opportunity Commission.

EEOC. (2012). Strategic Enforcement Plan, 2013-2016. Washington, DC: U.S. Equal Employment Opportunity Commission.

EEOC. (2012). United States Equal Employment Opportunity Commission Strategic Plan for Fiscal Years 2012-2016. Washington, DC: U.S. Equal Employment Opportunity Commission. http://www.eeoc.gov/eeoc/plan/strategic_plan_12to16.cfm

Federal Consulting Group, U.S. Department of Interior. (2010). U.S. Equal Employment Opportunity Commission Evaluation of the Priority Charge Handling Procedures Report. Washington, DC: U.S. Equal Employment Opportunity Commission.

Green, M.Z. (2001). Proposing a New Paradigm for EEOC Enforcement After 35 Years: Outsourcing Charge Processing by Mandatory Mediation. Dickinson Law Review, 105, 305-342.

Hersch, J. (2011). Compensating Differentials for Sexual Harassment. Paper presented at American Economic Association Annual Meeting, Denver, CO, January 8.

Hersch, J. (2013). Characteristics of Color Discrimination Charges Filed with the EEOC. In R.E. Hall (Ed.), The Melanin Millennium: Skin Color as 21st Century International Discourse, 231-246. Dordrecht, NL: Springer.

Igasaki, P.M. (2001). Doing the Best with What We Had: Building a More Effective Equal Employment Opportunity Commission During the Clinton-Gore Administration. The Labor Lawyer, 17(2), 261-284.

Igasaki, P.M., Miller, P.S. (1998). Priority Charge Handling Task Force and Litigation Task Force Report. Washington, DC: U.S. Equal Employment Opportunity Commission.

Kearney, R.A. (2001). Who's "In Charge" at the EEOC? Drake Law Review, 50(1), 1-31.

McAllister, S.B., Burbano, F., Christopher, G., Hamilton, M.R., White, B. (2003). Equal Employment Opportunity Commission: Organizing for the Future. A Report by a Panel of the National Academy of Public Administration for the U.S. Equal Employment Opportunity Commission. Washington, DC: National Academy of Public Administration.

McDermott, E.P., Jose, A., Obar, R., Bowers, M., Polkinghorn, B. (2002). Has the EEOC Hit A Home Run? An Evaluation of the EEOC Mediation Program from the Participants' Perspective. Advances In Industrial and Labor Relations, 11, 1-40.

McDermott, E.P., Obar. R. (2004). "What's Going On" in Mediation: An Empirical Analysis of the Influence of a Mediator's Style on Party Satisfaction and Monetary Benefit. Harvard Negotiation Law Review, 9, 75-114.

McDermott, E.P., Obar, R., Jose, A., Bowers, M. (2000). An Evaluation of the Equal Employment Opportunity Commission Mediation Program. Washington, DC: U.S. Equal Employment Opportunity Commission.

McDermott, E.P., Obar, R., Jose, A., Polkinghorn, B. (2001). The EEOC Mediation Program: Mediators' Perspective on the Parties, Processes, and Outcomes. Washington, DC: U.S. Equal Employment Opportunity Commission.

McGill, S., Tracey, A.M. (2011). Building a New Bridge over Troubled Waters: Lessons Learned from Canadian and U.S. Arbitration of Human Rights and Discrimination Employment Claims. Cardozo Journal of International and Comparative Law,20(1), 1-73. http://www.cjicl.com/uploads/2/9/5/9/2959791/cjicl_20.1_mcgill__tracey_article.pdf.

McKenna, M.A. (2005). Discriminatory Practices Charged Under EEOC: An Empirical Analysis of Investigated Complaints Filed by Those Who Have Cancer. Dissertation, University of Maryland College Park.

Modesitt, N.M. (2010), Reinventing the EEOC. Southern Methodist University Law Review, 63(4), 1237-1277.

Morra, L.G. (1995). Burgeoning Workload Calls for New Approaches. Testimony Before the Committee on Labor and Human Resources, U.S. Senate, May 23. GAO/T-HEHS-95-170. Washington, DC: U.S. General Accountability Office.
http://www.gao.gov/assets/110/106040.pdf

Moss, K., Burris, S., Ullman, M.D, Johnsen, M., Swanson, J.W. (2001). Unfunded Mandate: An Empirical Study of the Implementation of the Americans with Disabilities Act by the Equal Employment Opportunity Commission. Kansas Law Review 50(1), 1-109.

National Employment Lawyers Association. (2007). Worker's Rights in Jeopardy: EEOC's Enforcement of Equal Employment Opportunity Laws Impeded by Inadequate Funding. Washington, DC.

Occhialino, A.N., Vail, D. (2005). Why the EEOC (Still) Matters. Hofstra Labor & Employment Law Journal, 22(2), 671-708.
http://lawarchive.hofstra.edu/pdf/Academics/Journals/LaborAndEmploymentLawJournal/labor_Occhialino_Vail_vol22no2.pdf.

Robertson, R.E. (2005). Equal Employment Opportunity Commission: Actions Taken, but Agency Restructuring Efforts Could Benefit from a More Systematic Consideration of Advisory Panel's Recommendations. GAO-06-10. Washington, DC: U.S. General Accountability Office.

Selmi, M. (1996). The Value of the EEOC: Reexamining the Agency's Role in Employment Discrimination Law. Ohio State Law Journal, 57, 1-64.

Silverman, L.E., et al. (2006). Systemic Task Force Report. Washington, DC: U.S. Equal Employment Opportunity Commission.

Sternlight, J.R. (2004). In Search of the Best Procedure for Enforcing Employment Discrimination Laws: A Comparative Analysis. Tulane Law Review, 78, 1401-1499.

Zinc, D.L., Gutman, A. (2005). Statistical Trends in Private Sector Employment Discrimination Suits. In F.J. Landy (ed.), Employment Discrimination Litigation: Behavioral, Quantitative, and Legal Perspectives. San Francisco: Jossey Bass.

APPENDIX B: PRIORITY CHARGE HANDLING PROCESS (PCHP)

Under the Priority Charge Handling Process (PCHP), each incoming charge is assigned a priority status based on the information provided by the charging party (the alleged victim of discrimination) during an initial intake interview and on the perceived credibility of the charging party. Exhibit 1 details the steps involved in the current private sector charge processing system. First, the EEOC conducts a priority assessment of the charge and assigns it a priority status based on the likelihood of establishing that discrimination occurred. More specifically, Category A indicates that the charge falls within the national or local enforcement plan, or that further investigation is likely to lead to a cause finding. Category B indicates that additional evidence will be required to determine whether further investigation will reveal discrimination. Category C indicates that the charge will most likely be dismissed since the EEOC is unlikely to prove that discrimination occurred based on the evidence presented by the charging party. With this assessment from EEOC, the charging party must then decide whether to pursue the charge. Accordingly, the charge is settled either through mediation or through further investigation, which may result in conciliation or litigation.

Exhibit B1. Private Sector Charge Processing System

The PCHP was instrumental in reducing the pending charge inventory in the 1990s; by 2001, the inventory consisted of approximately 32,000 charges, and processing time was reduced to just over 6 months.⁽⁷⁰⁾ The inventory of pending charges, however, nearly tripled from FY 2002 (29,041)⁽⁷¹⁾ to FY 2010 (86,338)⁽⁷²⁾ for a variety of reasons. More recently, the EEOC has made progress in reducing the inventory. The year-end inventory was 70,312 in FY 2012 with the EEOC receiving 99,412 private sector workplace discrimination charges and resolving 111,139 charges in FY 2012.⁽⁷³⁾

APPENDIX C: FY 2012-2016 STRATEGIC PLAN

The EEOC is committed to serve the nation more efficiently, as emphasized in the Commission's FY 2012-2016 Strategic Plan.⁽⁷⁴⁾ The FY 2012-2016 Strategic Plan reflects a new strategy for implementing the power granted to the Commission by Congress in Title VII of theCivil Rights Act of 1964: "to prevent any person from engaging in any unlawful employment practice." In line with the Commission's primary mission to "stop and remedy unlawful employment discrimination," with a vision for "justice and equality in the workplace," the Strategic Plan is founded upon three values-commitment to justice, accountability, and integrity.⁽⁷⁵⁾ In keeping with these values, the plan prioritizes a coordinated approach to law enforcement and recognizes that operational silos have at times hampered the agency's efforts to prevent and remedy unlawful discrimination. To accomplish this, it outlines three objectives, each with its own set of outcome goals:

1. Combat employment discrimination through strategic law enforcement, with the outcome goals of (1) having a broad impact on reducing employment discrimination at the national and local levels, and (2) remedying discriminatory practices and securing meaningful relief for victims of discrimination.
2. Prevent employment discrimination through education and outreach, with the outcome goals of (1) better informing the public how to exercise their right to employment free of discrimination, and (2) informing employers, unions, and employment agencies (covered entities) how to better address and resolve Equal Employment Opportunity (EEO) issues, thereby creating more inclusive workplaces.
3. Deliver excellent and consistent service through a skilled and diverse workforce and effective systems,with the outcome goal of timely, informative, and high quality interactions with the public.

The FY 2012-2016 Strategic Plan departs from the previous plan by focusing less on measuring output and more on achieving long-term goals. As such, the plan requires the Commission to be proactive. This includes embarking on a year-long effort to create a conceptual framework that will inform, justify, and support the quantitative and qualitative performance measures identified in the plan.

APPENDIX D: EEOC LEADERSHIP AND STAFF INTERVIEW GUIDE

Thank you for agreeing to speak with us today. I am Manan Roy from IMPAQ International and I have Dr. Steven Garasky, Vice President, Human Services with me on the phone. As you may be aware, IMPAQ International has been hired by the EEOC to conduct a Review of Evaluations to identify significant new and useful conclusions and recommendations regarding EEOC's private sector enforcement activities.

Through this Review of Evaluations project, we aim to answer a few research questions such as:

• What factors are associated with the EEOC efficiently carrying out private charge processing activities?
• What factors are associated with the EEOC effectively carrying out private charge processing activities?
• What are the key information gaps related to developing strategies to more efficiently and effectively carry out private charge processing activities?

The purpose of this interview is to learn from your experience about the efficiency and effectiveness of EEOC regarding its private sector enforcement activities.

The information you provide to us will be used in combination with what we learn from others, in addition to the studies we have reviewed, to document recommendations and useful conclusions regarding EEOC's private sector enforcement activities.

Your comments are confidential and you will not be identified by name in any report. Steve and I will be taking notes while we talk. With your permission, we would also like to record this session so that we can refer to the audio to clarify our notes later if necessary. Do we have your permission to begin recording? Thank you.

Do you have any questions before I begin?

1. Background and Context

2. We'd like to start with some general questions to give us a little bit of background.

   • What is your current position and how long have you been at EEOC?
   • Can you describe your individual role and job responsibilities?
3. Project-Related Questions

4. Now, we'd like to discuss your opinion and thoughts about the private sector enforcement activities of the EEOC

   • In your current capacity and based on your experience, what according to you are the strengths of the EEOC's private sector enforcement activities?
   • In your current capacity and based on your experience, what are your concerns about the EEOC's current private sector enforcement activities?
   • What would be your practical recommendations to address these concerns?
   • What would be your practical recommendations to make the private sector enforcement activities more efficient?
   • What would be your practical recommendations to make the private sector enforcement activities more effective?
   • Are there any other studies that you would recommend us to include in the review?

APPENDIX E: CONVERSATION WITH DISTRICT DIRECTORS PROTOCOL

Thank you for agreeing to speak with us today. I am Dr. Steven Garasky, Vice President for Human Services Research at IMPAQ International. Joining me is Dr. Manan Roy, also from IMPAQ. As you are aware, our firm has been hired by the EEOC to conduct a review of evaluations of EEOC's private sector enforcement activities. The goal of the project is to provide recommendations to enhance effectiveness and efficiency in this area.
The purpose of this conversation is to obtain your thoughts regarding conclusions we have reached that will inform recommendations we are considering. The information you provide will be used in combination with what we have learned from other sources. Please know that your comments are confidential and you will not be identified by name in any report.
We will be taking notes while we talk, but with your permission, we would also like to record this session so that we can refer to the audio to clarify our notes later, if necessary. The audio files will not be shared with anyone outside of IMPAQ and will be destroyed at the completion of the project. Do we have your permission to begin recording?
Do you have any questions before I begin?

1. Backgroundand Context

2. We'd like to start with some general questions to give us a little bit of background.

   • All three of you are Field Directors. Is that correct?
   • Can each of you tell us how long have you been with the EEOC and describe your role and responsibilities?
3. Project-Related Questions

4. Now, we'd like to discuss the private sector enforcement activities of the EEOC. You received a two-page document derived from our draft report from Larkin Jennings of the Office of the Inspector General. Mr. Jennings is the Contracting Officer's Technical Representative for this project. The first page of the document was an attempt to give you some context for the study in terms of our approach. We'd be happy to discuss our methodology further if that would be beneficial, but would especially like to focus on the second page which is a broad overview of our conclusions based on the numerous studies we reviewed and interviews we conducted with EEOC leadership and staff.

   • To begin, do you have any questions about the purpose of the project or the methodology we used?

   Shifting to our conclusions, we focused on three elements of private sector enforcement that offered in our review and interviews a relative consensus regarding areas in which changes could be made to enhance efficiency and/or effectiveness. These areas are identifying a significant charge, improving the consistency of intake procedures, and improving data collection. We would like your thoughts on our assessments in each of these areas. Again, we know that the materials you received provide a very broad overview and limited context for these conclusions. Nevertheless, we believe that your feedback will be very beneficial to the study.
   We'd like this to be a free flowing conversation. Feel free to provide overall comments, identify where you agree and disagree with our thoughts, etc.

   • Let's begin with identifying a significant charge. Again, for efficiency, rather than our reading the materials provided to you, please jump right in with your thoughts.
   • Let's move to intake procedures. Again, please jump in with your thoughts on this area.
   • Lastly, let's talk about data collection. Do you have any comments on that paragraph?
5. Closing Remarks

6. Thank you again for taking the time to speak with us. We truly appreciate your thoughtful remarks. Please do not hesitate to reach out to me if you have further questions or additional thoughts or feedback. I can be reached at 443-539-2087. Have a good afternoon. Good-bye.

APPENDIX F: VALIDITY OF THE SELECTED STUDIES

We conclude that the studies included in the review are suitable and valid in relation to the research questions they seek to answer. Importantly, the breadth of topics addressed by the studies spanned a wide variety of research questions and sub-questions related to the organization, functioning, responsibilities, types of charges, and activities performed by EEOC.

A key factor in assessing validity was the reputation of the research outlet. Many studies were articles drawn from a range of peer-reviewed publications including the following:

• Harvard Negotiation Law Review
• Tulane Law Review
• Hofstra Labor & Employment Law Journal
• Drake Law Review
• Kansas Law Review
• Dickinson Law Review
• Temple Political & Civil Rights Law Review
• Ohio State Law Journal
• Cardozo Journal of International and Comparative Law
• Southern Methodist University Law Review
• Journal of the American Economic Association

Other studies were obtained from nationally renowned research organizations, for example, the ILR School Employment and Disability Institute at Cornell University and the National Academy of Public Administration. Some studies were conducted by external contractors to EEOC including Dr. E. Patrick McDermott and his co-authors; Federal Consulting Group; National Employment Lawyers Association; and Virginia Commonwealth University. Some of the studies were conducted by EEOC staff.
Another factor in assessing validity was study methodology. The methodologies adopted in the reviewed studies included the following:

• Review of EEOC documents and legal documents
• Review of previous studies
• Analysis of IMS and CPS databases for EEOC strengths and weaknesses
• Surveys conducted at mediation meetings
• Focus groups conducted with participants (charging parties, respondents, and EEOC mediators)
• Interviews with EEOC staff and EEOC customers (charging parties and respondents).

Study geography was important to assessing validity as well. The selected studies included some that were national in scope, others that covered most EEOC field offices, and still others that focused on special sub-populations, such as individuals with disabilities, individuals with HIV/AIDS, and cancer patients.

Finally, although the project team reviewed many studies dating back to the 1990s, this evaluation synthesis report emphasizes studies published after 2003. We referred to some of the older studies where they were deemed pertinent.

APPENDIX G: COMMENTS FROM THE OFFICE OF THE CHAIR

U.S. EQUAL EMPLOYMENT OPPORTUNITY COMMISSION
Washington, D.C. 20507

Office of the Chair

March 21, 2013

TO:	Milton Mayo Inspector General
FROM:	Claudia A. Withers Chief Operating Officer
RE:	Response to OIG Draft Report on the Review of Evaluations

Thank you for the opportunity to comment on the draft report on the review of evaluations of the EEOC's charge processing activities. The draft states that its purpose is to provide EEOC leadership and stakeholders with recommendations to enhance the efficiency and effectiveness of EEOC private sector enforcement activities, including mediation. We reviewed your recommendations with great interest, and we are in general agreement. Indeed, the robust agency wide strategic planning and implementation effort in which we have been engaged since July of 2011 is very much consistent with the purpose and recommendations in the report. We appreciate the independent evaluation of the Office of Inspector General, and we are pleased that our efforts to date appear to be in sync with this evaluation.

The report made the following specific recommendations to the agency:

1. The EEOC should further standardize intake procedures across field offices.
2. The EEOC should document criteria for determining category C charges
3. The EEOC should develop a national approach for addressing systemic discrimination
4. The EEOC should continue to review the range of information obtained during intake interviews and how it is stored in the IMS.
5. The EEOC should investigate the merits of expanding the information it obtains related to hiring and terminations.

EEOC' s FY 2012-2016 Strategic Plan

The mission for the agency under the Strategic Plan is to Stop and Remedy Unlawful Employment Discrimination. The Vision is: Justice and Equality in the Workplace.

The plan contains three strategic objectives:

1. Combat employment discrimination through strategic law enforcement
2. Prevent employment discrimination through education and outreach
3. Deliver excellent service through effective systems, updated technology and a skilled and diverse workforce

The plan includes a number of outcome goals, strategies and performance measures. Based on our review of the draft report in the context of our strategic plan, it is clear that the report's recommendations are contemplated in our implementation of the strategic plan.

• The Strategic Plan's Performance Measure 2calls for the development of a quality control plan, which will establish criteria for quality investigations and conciliations. The Quality Control Plan workgroup, which includes EEOC employees from across the agency, is engaged in discussions which should include issues involving intake, the data that is included in IMS, and the issues of consistency of service across the agency. Accordingly, the draft report's recommendations will be a part of the discussion.
• Performance Measure 4of the Strategic Plan builds on EEOC's 2006 Systemic Task Force Report, and requires that the agency develop a baseline for systemic litigation as a percentage of the litigation filed by the agency. Such a baseline has been set, and the agency is working to meet the baseline metrics.
• Performance Measure 1of the Strategic Plan calls for the development of a Strategic Enforcement Plan. Such a plan was approved by the Commission in December 2012. Since that time, we have been engaged in implementation of the Strategic Enforcement Plan.

• Among other things, the SEP outlined national priorities:

  • Eliminating barriers in recruitment and hiring;
  • Protecting immigrant, migrant, and other vulnerable workers;
  • Addressing emerging and developing issues;
  • Enforcing equal pay laws;
  • Preserving access to the legal system; and
  • Preventing harassment through systemic enforcement and targeted
    outreach

The SEP calls for the development of teams corresponding to the above-named priorities, to recommend national strategies for effectively addressing each priority. The strategies under consideration will span all of EEOC's available tools, including administrative and legal enforcement, outreach and communications, data collection and research, and policy development throughout the private, public and federal sectors.

Finally, the SEP also calls for the development of a multi-year Research and Data Plan in FY 2013 which will focus on the SEP national priories. Work has begun on the development of the plan, which can include a discussion of Recommendation 5 in the draft report.

Thank you for the opportunity to comment on the draft report. We appreciate the independent assessment of our work, and are gratified that the report's recommendations are congruent with the work we are doing to fulfill the agency's mission.

131 M Street, N. E., Suite 6NW08F
Phone (202) 663-4001
TTY (202) 663-4141
FAX (202) 663-4110
JACQUELINE.BERRIEN@EEOC.GOV

APPENDIX H: COMMENTS FROM THE OFFICE OF FIELD PROGRAMS

U.S. EQUAL EMPLOYMENT OPPORTUNITY COMMISSION
Washington, D.C. 20507

Office of Field Programs

March 21, 2013

MEMORANDUM

TO:	Milton Mayo Inspector General
FROM:	Nicholas M. Inzeo, Director Office of Field Programs
SUBJECT :	Comments on Draft Report, "Review of Evaluations" (OIG Report Number 2012-09-REV), prepared by IMPAQ International

This is in response to your request for review of the draft report, "Review of Evaluations," prepared by IMPAQ International, LLC. We have completed our review and offer the following comments.

The draft report examines studies, law review articles, and reports about the EEOC in order to synthesize the information to identify recommendations to improve the efficiency and effectiveness of the private sector charge process, including mediation. Most of the studies were published before 2011. The draft report relies in particular on two consultant-prepared evaluations of the charge process (Development Services Group, "Evaluation of Intake and End-of- Fiscal-Year Closures of the EEOC Private Sector Charge Process," 2006; and Federal Consulting Group, "U.S. Equal Employment Opportunity Commission Evaluation of the Priority Charge Handling Procedures Report," 2010), and on the EEOC's own 2006 Systemic Task Force Report, which made recommendations that were adopted and have been largely implemented by the Commission in its systemic program. We appreciate this summary and are pleased that the draft report acknowledges the accomplishments of the EEOC's mediation program, citing several positive studies and noting that these studies raised no "information gaps or concerns" about mediation.

The Commission's adoption last year of the Strategic Plan for FY 2012 - 2016 and the Strategic Enforcement Plan introduced a comprehensive set of new strategies and mechanisms to strengthen the effectiveness and efficiency of the agency's work in all areas. In addition, the draft report contains several recommendations that we agree with and have previously implemented. With respect to the private sector charge process, we have taken a number of actions in the years since issuance of the reports and evaluations that the draft report cites, in order to accomplish much of what the draft report is recommending. Currently, we are focusing our efforts on implementation of the Strategic Plan and Strategic Enforcement Plan. These efforts are consistent with the draft report's observations, but we note that the situation at the agency is different now than when the cited reports and evaluations were written several years ago.

We have the following comments on selected portions of the draft report:

• At the top of page 2, the report states that the EEOC received 99,412 private sector charges in FY 2012 but that "the inventory of charges has more than doubled over the last decade." This static comparison is less informative than an examination of the current trend in charge intake and inventory. The draft report neglects to mention the significant reductions in inventory that have been achieved in the last two fiscal years, for an overall 20% reduction in inventory by the end of FY 2012. Also, the report in footnotes 7 and 69 cites to the EEOC's Performance and Accountability Report (PAR) Highlights for FY 2011. We suggest that the draft report reflect the most recent PAR, FY 2012, and utilize as its source the full report, which is available on our web site. These reports for the last several fiscal years provide details of many of the steps that have been taken to implement the Systemic Task Force Recommendations and make improvements to charge processing, which address not only the issues raised in the earlier studies and evaluations cited in the draft report, but also the recommendations contained in this draft report.
• With respect to the draft report's recommendation on improving the consistency of intake procedures, the report primarily relies upon the 2006 evaluation by the Development Services Group (DSG). The draft report does not seem to capture what was done in response to or after the DSG report and other evaluations. In the draft report at page 7, the description of some "best practices" as occurring in only some field offices is inaccurate; in fact, all offices follow these intake practices. We have taken many actions since 2006, including development of uniform questionnaires, brochures and checklists; intake and interviewing skills training; development and refinement of the intake questionnaire; and refresher training for all field enforcement staff on PCHP. Moreover, one of the key components of the current Strategic Plan is the development of a Quality Control Plan, which will establish criteria for quality investigations and conciliations. This work is well underway.
• In the section on addressing systemic discrimination, the draft report recognizes that the EEOC's FY 2012 - 2016 Strategic Plan and Strategic Enforcement Plan signify EEOC's commitment to pursuing systemic investigations. However, the balance of this section primarily repeats the findings of the EEOC's 2006 Systemic Task Force Report. For example, while acknowledging some "laudable" recent efforts to address systemic discrimination, the draft report concludes that "evidence indicates" that the EEOC "does not identify systemic discrimination in a consistent and coordinated manner" and that "there is a need for establishing a national approach to combat systemic discrimination." Those were the very findings of the Task Force Report. When the Task Force Report was issued in April 2006, the Commission simultaneously adopted its numerous recommendations to launch a concerted nationwide effort to identify and remedy systemic discrimination. The Strategic Plan and Strategic Enforcement Plan build on what the EEOC has accomplished since 2006, and the accomplishments reported in the PAR demonstrate the progress that has been made.
• With respect to systemic enforcement activities, the draft report further states that "Steps toward a national approach should begin with district reports to EEOC headquarters detailing how each district is identifying and addressing systemic discrimination," and that these reports should be updated regularly. In fact, this was a recommendation of the 2006 Systemic Task Force. It was adopted by the Commission and has been a routine part of the agency's systemic program since that time. District Office systemic plans were submitted in the first quarter of FY 2007 and, as required by the Task Force recommendations, provided detailed information on how the district was identifying and addressing systemic discrimination, with reference to specific focus areas and industries, targets, potential and pending litigation, and outreach and training efforts. District Office reporting on systemic investigations and litigation, to the General Counsel and the Office of Field Programs, has continued since that time on a quarterly basis.
• On page 10, the report references limitations on data in IMS but the referenced evaluation which serves as the basis for this finding was published in 2001 and was based on the agency's previous data system (Charge Data System-CDS) and not IMS. The IMS significantly expanded not only the data collection capacity for the agency on its charge processing activity in general but also provides a search capacity for prior charge activity involving specific Charging Parties and Respondents.

If you have any questions about our comments, please contact me, or Sharon Shoemaker or Sue
Murphy of my staff.

cc: Claudia Withers, Chief Operating Officer

Deidre Flippen, Director
Office of Research, Information and Planning

APPENDIX I: COMMENTS FROM THE OFFICE OF RESEARCH, INFORMATION, AND PLANNING

Response from Deidre Flippen, Director, Office of Research, Information, and Planning, EEOC

We appreciate the opportunity to comment on the Review of Evaluations - OIG Report Number 2012-09-REV). We hope our comments are useful.

Generally, we believe this evaluation suffers from two key limitations in its methodology. First, the report is not really research per se but a review of relevant literature that is generally used to develop research and generally not used to make recommendations. While such "reviews" can be helpful they are generally carefully structured as meta analyses and data is extracted, evaluated and often regenerated to produce more quantitative results. This report lacks that type of rigor and as a result may make recommendations that are not empirically based. The study reports a number of measures of association (page ii). Associations are generally weaker level relationships and might be considered inappropriate measures for program evaluations. Instead some measure of causality is preferred before programs modify their practices in response to an evaluation. "Systemic evaluation of program impact incorporates the elements of causality by establishing covariation and time order and by eliminating all possible spurious and confounding influences" (Discovering Whether Programs Work: A Guide to Statistical Methods for Program Evaluation, Laura Irwin Langbein, p. 41.)

These measurement issues may be inherent in these types of "reviews" rather than primary research. However, the problem is greatly exacerbated by the literature selected for review. The articles cited do not appear to be empirically based in most instances. The sources used are surprising over represented with law reviews rather than academic journals from the social sciences. Our office works with a number of researchers who have used our survey and charge data to examine a number of issues relevant to the statutes we enforce and EEOC activities but not a single one of these researchers is used in this review.

The study lacks specificity that would be important for EEOC decision makers. For example on page 7, the study calls for greater standardization but provides no concrete examples of inconsistencies and how these inconsistencies are problematic.

---

FOOTNOTES: