City of Dallas

Request for Competitive Sealed Proposals

For

Promotional Examination Administrator

BRZ1705

SECTION 1 –BACKGROUND

1.1Purpose

The City of Dallas (hereinafter, “the City”) seeks proposals from qualified vendors to develop and administer the promotional examination processes for the Dallas Fire Recue Department and the Dallas Police Department for a term of thirty-six (36) months. These promotional examinations will be based on existing comprehensive job analyses. The specifications are detailed below by categories, Group 1 – Fire and Group 2 – Police. Vendors may bid on Group 1, Group 2, or both.

1.2Background – General

The Police and Fire Departments are not unionized. However, the City does have a meet and confer agreement with the Police and Fire Associations. There are no current issues pertaining to testing and no pending litigation or consent decrees at this time involving City of Dallas tests.

1.3Vendor Requirements

Vendor will provide all personnel, equipment, tools, materials, supervision, and other items and non-personnel services necessary to conduct the promotional examination for each rank listed. Vendor shall utilize employees who have the qualifications, education, experience, training and certifications necessary to provide all services as described in this RFCSP. Vendor shall dedicate a sufficient number of employees to provide all services in the manner required by and within the time requirements of the contract. The vendor shall conduct an initial kick-off meeting with the City to present a Plan of Action and Milestones for project execution within two weeks of date of contract execution. Vendor shall be available to conduct weekly conference calls or in-person meetings as requested by the City.

The City will be responsible for obtaining hotel accommodations for evaluators and vendor, catering, and transportation from the hotel to the assessment center facility, if applicable.

SECTION2– STATEMENT OF EXPERIENCE

2.1 Experience

The qualifications and operational capabilities of prospective vendors are fundamental criteria which the City will assess in selecting the most advantageous Proposer(s).

Vendor shall have all of the following qualifications:

  • A minimum of five years’ experience in the field of Industrial-Organizational Psychology with particular emphasis on designing and conducting promotional examinations in fire/police department settings.
  • Experience conducting public safety promotional testing for large governmental agencies similar to the City of Dallas, TX.
  • Experience in designing written exams and assessment centers for Fire and/or Police Department positions.
  • Experience in statistical analyses.
  • Experience facilitating group meetings (e.g., focus groups, orientations).
  • An understanding of the requirements and professional standards as outlined in the Uniform Guidelines on Employee Selection Procedures, Principles for the Validation and Use of Personnel Selection Procedures, and other rules and regulations that govern testing.
  • Experience developing testing instruments based on job analysis results.
  • Experience working with concepts, practices, and procedures in the area of test development.
  • Experience developing and validating examinations including multiple-choice written examinations, assessment centers and other employment tests.
  • Experience writing formal written exam and assessment center reports.
  • Experience providing expert testimony in fire/police cases.

Vendor shall be available, throughout the term of the contract, to meet with the City, in person, five to ten times at the discretion and request of the City.

SECTION 3 – FIRE SPECIFICATIONS (GROUP 1)

3.1Background

The City of DallasCivil Service Department develops and administers entry level and promotional examinations for the Dallas Fire Rescue Department. The Fire Operations uniformed ranks for which there are Civil Service examinations include Fire Rescue Officer Trainee, Driver Engineer, Lieutenant, Captain, and Battalion Chief. The Fire Prevention uniformed ranks for which there are Civil Service examinations include Fire Prevention Officer Trainee, Senior Fire Prevention Officer, Lieutenant, Captain, and Section Chief. Classification specifications for these ranks may be found in the attached document,Fire Class Specifications (Operations and Prevention). The classification specifications describe the purpose of the position, the essential functions, the KSAOs required to perform the work, special requirements, and the working conditions and hazards.

The upcoming examinations for the FireRescueranks will be based on results from comprehensive job analyses. The City has determined the need to conduct promotional examinations for all Fire Rescue ranks in 2017and all Fire Prevention ranks in 2018. The current Civil Service selection strategies implemented for the four lowest ranks (Fire Rescue Officer Trainee, Driver Engineer, Fire Prevention Officer Trainee, Senior Fire Prevention Officer) use 100 item multiple-choice written examinations only. The promotional strategies for the remaining ranks employ multiple-choice written examinations and assessment centers. Historically, the City has used three to four exercises in the assessment centers. This strategy will be continued to provide continuity between all ranks being tested in 2017. The Fire Rescue Officer Trainee and the Fire Prevention Officer Trainee are not a part of this RFCSP. The minimum qualifications for each rank are described in the Civil Service Rules which are located at

The numbers of candidates who participatedin the last round ofpromotional processes for each rank are listed below.

ClassificationWritten ExamAssessment Center

Fire Operations Battalion Chief (2013) 3831

Fire Operations Captain (2013) 9580

Fire Operations Lieutenant (2014) 173119

Driver Engineer (2014) 507N/A

Fire Prevention Section Chief (2012) 53

Fire Prevention Captain (2012) 117

Fire Prevention Lieutenant (2012) 2715

Senior Fire Prevention Officer (2013) 18N/A

The traditionally used assessment center exercises for each rank are listed below.

ClassificationAssessment Center Exercises

Fire Operations Battalion Chief (2013)In-basket, operational exercise, and oral presentation

Fire Operations Captain (2013)Operational, personnel, and policy exercises

Fire Operations Lieutenant (2014)Operational, personnel, and policy exercises

Fire Prevention Section Chief (2012)In-basket, operational, personnel, and policy exercises

Fire Prevention Captain (2012)Operational, personnel, and policy exercises

Fire Prevention Lieutenant (2012)Operational, personnel, and policy exercises

Historically, the Fire Prevention assessment centers for Lieutenant, Captain, and Section Chief are administered in this order during the same week, one day for each rank. Evaluators are trained together and shared appropriately (e.g. evaluators are at or above the rank tested).

Target test dates (based on testing location availability) for each rank is listed below. Facilities will be provided by and paid for by the City.

ClassificationWritten ExamAssessment Center

Fire Operations Battalion Chief TBD*TBD

Fire Operations Captain 5/9/2017**7/8/2017-7/12/2017

Fire Operations Lieutenant7/6/2017TBD

Driver Engineer September/October 2017N/A

Fire Prevention Section Chief Calendar Year 2018Calendar Year 2018

Fire Prevention CaptainCalendar Year 2018Calendar Year 2018

Fire Prevention Lieutenant Calendar Year 2018Calendar Year 2018

Senior Fire Prevention OfficerCalendar Year 2018N/A

*Examination may be provided by City of Dallas Civil Service Department. Vendor should still provide pricing for examination.

**The City will send out any required notices forthe May 9, 2017 examination.

SECTION 4 – PROJECT UNDERSTANDING AND SCOPE

4.1Objectives

The purpose of the promotional examination process is to

  • Develop valid examination instruments (one-hundred item multiple choice examinations and assessment centers as applicable) for the ranks of Fire Operations Battalion Chief, Operations Captain, Operations Lieutenant, Driver Engineer, Fire Prevention Section Chief, Prevention Captain, Prevention Lieutenant, and Senior Fire Prevention Officer.
  • Allow the City to promote individuals based on essential KSAOs required for the job.
  • Help engender the community’s confidence in the City and Fire Rescue Department by maintaining a knowledgeable and qualified workforce.

4.2 Scope of Work

  • The scope of the project is limited to employees within the City. These employees will serve as the subject matter experts (SME)for the various stages of the test development process (e.g., job observations, focus groups, item reviews, etc.),and the Vendor will be expected to consult with the SMEs.
  • Provide evaluators for assessment centers. The evaluatorswill be at or above the level being assessed and will be from cities comparable to the City of Dallas in size.
  • Provide a proposed project plan with timelines for each rank listed. The timelines shall include estimated time span for each step of the test development process, estimated incumbent and supervisor time requirements for each step requiring SME participation.
  • Vendor shall conduct the test development in compliance with applicable laws and professional standards of the testing industry.
  • Develop the selection instruments. The test development process shall include:
  • Conducting field observations and/or focus group meetings to gain the required understanding of the position.
  • Providing a candidate orientation video to help candidates prepare for the written examination and a mechanism for answering candidates’ questions. (60 days prior to test)
  • Developing all multiple-choice items for the 100-item written exam.
  • Conducting SME reviews of items to ensure item quality, relevance, and content validity.
  • Constructing final exam and creating alternate forms and answer keys as needed.
  • Conducting a pass-point session for the written exam, using the modified Angoffor other appropriate method, to determine a legally defensible cut score.
  • Conducting written exam challenge process (4 days of challenges, 2 days of comments, 1 day challenge review panel). This includes writing staff responses to challenged test items.
  • Providing a minimum of three separate days for on-site candidate orientations for the assessment center process for each rank tested to accommodate three shifts.
  • Obtainingexternal evaluatorsfrom comparable cities to participate in the assessment center.
  • Developing the assessment center exercises.
  • Conducting SME reviews of the assessment center exercises to ensure exercise quality, relevance and content validity.
  • Developing behaviorally anchored rating scales for each assessment center exercise.
  • Providing on-site evaluator training prior to the administration of the assessment center to include at a minimum the training agenda, descriptions of the dimensions assessed, behavioral observation training, common rating errors, and mock exercise evaluation for calibration purposes.
  • Schedulingcandidates and evaluators for the assessment center administration.
  • Administering the assessment center over a three day period (one exercise per day).
  • Providing results of the assessment center process to candidates including a combined written exam and assessment center score using the weights outlined by the job analysis.
  • Ensuring test security before during and after the assessment center.

More detailed descriptions of the test development process can be found in the attached document,Promotion Exam Activities.

  • Focus group meetings and orientation sessions shall be held in City offices. They cannot be conducted via webinar or phone conference. The City will provide assistance in coordinating SME participation and securing meeting facilities.
  • Conduct statistical analyses on examination process.
  • Vendor shall provide a written exam and assessment center reportas applicable for each rank.
  • Vendor shall provide price quotes for each rank to be assessed and each component of the testing process (written exam and assessment center). Refer to the attached document, Price Sheet – Fire.
  • One quote for each of the 8 written exams listed in section 3.1.
  • One quote for each of the 6 assessment centers listed in section 3.1.
  • One quote for each of the 8 ranks listed in section 3.1.
  • The City of Dallas reserves the right to award contracts for any of the ranks listed or any individual test component.

SECTION 5 – POLICE SPECIFICATIONS (GROUP 2)

5.1Background

The Police uniformed ranks for which there are Civil Service examinations include Police Officer Trainee, Senior Corporal, Sergeant, and Lieutenant. Classification specifications for these ranks may be found in the attached document, Police Class Specifications. The classification specifications describe the purpose of the position, the essential functions, the KSAOs required to perform the work, special requirements and the working conditions and hazards.

The upcoming examinations for the Police ranks will be based on results from comprehensive job analyses. The City has determined the need to conduct promotional examinations for Police ranks in 2017. The current Civil Service selection strategies implemented for the two lowest ranks (Police Officer Trainee and Senior Corporal) use 100 item multiple-choice written examinations only. The promotional strategies for the remaining ranks employ 100 item multiple-choice written examinations and assessment centers. Historically, the City has used three to four exercises in the assessment centers. This strategy will be continued to provide continuity between all ranks being tested in 2017. The Police Officer Trainee is not a part of this RFCSP. The minimum qualifications for each rank are described in the Civil Service Rules which are located at

The numbers of candidates who participated in the last round of promotional processes for each rank are listed below.

ClassificationWritten ExamAssessment Center

Police Lieutenant (2013) 12494

Police Sergeant (2014) 226153

Police Senior Corporal (2015) 579NA

The traditionally used assessment center exercises for each rank are listed below.

ClassificationAssessment Center Exercises

Police Lieutenant (2013)In-basket, operational exercise, and oral presentation

Police Sergeant (2014)In-basket, operational, and personnel exercises

Police Senior Corporal (2015)N/A

Target test dates (based on testing location availability) for each rank is listed below. Facilities will be provided by and paid for by the City.

ClassificationWritten ExamAssessment Center

Police LieutenantApril 25, 2017June 10-14, 2017

Police SergeantMay 22, 2017August 14-18, 2017

Police Senior CorporalTBDTBD

SECTION 6– PROJECT UNDERSTANDING AND SCOPE

6.1Objectives

The purpose of the promotional examination process is to

  • Develop valid examination instruments (one-hundred item multiple choice examinations and assessment centers as applicable) for the ranks of Police Lieutenant, Sergeant and Senior Corporal.
  • Allow the City to promote individuals based on essential KSAOs required for the job.
  • Help engender the community’s confidence in the City and Police Department by maintaining a knowledgeable and qualified workforce.

6.2 Scope of Work

  • The scope of the project is limited to employees within the City. These employees will serve as the subject matter experts (SME) for the various stages of the test development process (e.g., job observations, focus groups, item reviews, etc.), and the Vendor will be expected to consult with the SMEs.
  • Provide evaluators for assessment centers. The evaluators will be at or above the level being assessed and will be from cities comparable to the City of Dallas in size.
  • Provide a proposed project plan with timelines for each rank listed. The timelines shall include estimated time span for each step of the test development process, estimated incumbent and supervisor time requirements for each step requiring SME participation.
  • Vendor shall conduct the test development in compliance with applicable laws and professional standards of the testing industry.
  • Develop the selection instruments. The test development process shall include:
  • Conducting field observations and/or focus group meetings to gain the required understanding of the position.
  • Providing a candidate orientation video to help candidates prepare for the written examination and a mechanism for answering candidates’ questions. (60 days prior to test)
  • Developing all multiple-choice items for the 100-item written exam.
  • Conducting SME reviews of items to ensure item quality, relevance, and content validity.
  • Constructing final exam and creating alternate forms and answer keys as needed.
  • Conducting a pass-point session for the written exam, using the modified Angoff or other appropriate method, to determine a legally defensible cut score.
  • Conducting written exam challenge process (4 days of challenges, 2 days of comments, 1 day challenge review panel). This includes writing staff responses to challenged test items.
  • Providing a minimum of three separate days for on-site candidate orientations for the assessment center process for each rank tested to accommodate three shifts.
  • Obtaining external evaluators from comparable cities to participate in the assessment center.
  • Developing the assessment center exercises.
  • Conducting SME reviews of the assessment center exercises to ensure exercise quality, relevance and content validity.
  • Developing behaviorally anchored rating scales for each assessment center exercise.
  • Providing on-site evaluator training prior to the administration of the assessment center to include at a minimum the training agenda, descriptions of the dimensions assessed, behavioral observation training, common rating errors, and mock exercise evaluation for calibration purposes.
  • Scheduling candidates and evaluators for the assessment center administration.
  • Administering the assessment center over a three day period (one exercise per day).
  • Providing results of the assessment center process to candidates including a combined written exam and assessment center score using the weights outlined by the job analysis.
  • Ensuring test security before during and after the assessment center.

More detailed descriptions of the test development process can be found in the attached document, Promotion Exam Activities.

  • Focus group meetings and orientation sessions shall be held in City offices. They cannot be conducted via webinar or phone conference. The City will provide assistance in coordinating SME participation and securing meeting facilities.
  • Conduct statistical analyses on examination process.
  • Vendor shall provide a written exam and assessment center report as applicable for each rank.
  • Vendor shall provide price quotes for each rank to be assessed and each component of the testing process (written exam and assessment center). Refer to the attached document, Price Sheet – Police.
  • One quote for each of the 3 written exams listed in section 5.1.
  • One quote for each of the 2 assessment centers listed in section 5.1.
  • One quote for each of the 3 ranks listed in section 5.1.
  • The City of Dallas reserves the right to award contracts for any of the ranks listed or any individual test component.

SECTION 7– OUTLINE FOR RFCSP SUBMITTAL