REQUEST FOR PROPOSAL NO.STATE-RFP000128

PENNSYLVANIA DEPARTMENT OF STATE

BUREAU OF PROFESSIONAL AND OCCUPATIONAL AFFAIRS

REAL ESTATE COMMISSION EXAMINATIONS

TECHNICAL SUBMITTAL

I-1.Statement of the Project.

Objectives:

The Commonwealth, Department, and Commission require the services of a qualified professional testing organization to: (1) develop, prepare, administer, via a computerized (electronic) testing system, and score two valid, reliable and legally defensible licensing examinations for applicants for licensure; and (2) perform examination-related services for a five-year fixed term beginning on the effective date.

The first examination is a broker examination for Real Estate Brokers (broker examination). The second examination is a salesperson examination for Real Estate Salespersons, Cemetery Brokers, Builder/Owner Salespersons and Rental Listing Referral Agents (salesperson examination). Both examinations contain a national and state portion. Candidates can take the national and state portions of the examination(s) separately.

The Offeror should describe how it will perform the tasks and deliver the services detailed below in I-5so that the objectives will be achieved at the highest level of quality and effectiveness.

The specific objectives are:

  1. To obtain a comprehensive examination administration program; and
  1. To obtain valid, reliable and legally defensible licensing examinations.

Nature and Scope of Project:

This RFP encompasses all services necessary for the development, preparation, administration and scoring of two examinations: a broker examination and a salesperson examination. The selected Offeror shall furnish all labor, materials, supplies, facilities, personnel, training, data processing services and security required to perform the necessary services. The necessary examinations must be delivered year-round at locations throughout Pennsylvania, as described more fully below.

This RFP sets forth the Commission’s requirements and its understanding of examination development and administration. If the Offeror desires to propose a different method to accomplish the goals of this RFP, while still meeting the Commission’s requirements, it may do so. However, the Offeror must fully explain any deviations from the tasks and services set forth in this RFP. The deviations and explanation shall be contained in a separate Appendix to the proposal. The Appendix should be numbered as Appendix B, Alternate Method and the cost should not deviate from the Cost Submittal provided through the Jaggaer system.

Offeror Response

I-2.Qualifications.

  1. Company Overview. The Commonwealth, Department, and Commission require the services of a qualified professional testing organization to: (1) develop, prepare, administer, via a computerized (electronic) testing system, and score two valid, reliable and legally defensible licensing examinations for applicants for licensure; and (2) perform examination-related services.

Offeror Response

  1. Prior Experience. Include experience inthe development, preparation, administration and scoring of examinations: a broker examination and a salesperson examination. Experience shown should be work done by individuals who will be assigned to this project as well as that of your company. Studies or projects referred to must be identified and the name of the customer shown, including the name, address, and telephone number of the responsible official of the customer, company, or agency who may be contacted.

Offeror Response

  1. Personnel. Describe in narrative form the number of executive and professional personnel who will be engaged in the work and indicate where these personnel will be physically located during the time they are engaged in the Project. For key personnel, include the employee’s name, and through a resume or similar document, the Project personnel’s education and experience in the development, preparation, administration and scoring of examinations: a broker examination and a salesperson examination. The selected Offeror shall employ or contract with a qualified psychometrician to assist with test development and evaluation. Indicate the responsibilities each individual will have in this Project and how long each has been with your company.

Offeror Response

  1. Subcontractors: Provide asubcontracting plan for all subcontractors, including small diverse business and small business subcontractors, who will be assigned to the Project. The selected Offeror is prohibited from subcontracting or outsourcing any part of this Project without the express written approval from the Commonwealth. Upon award of the contract resulting from this RFP, subcontractors included in the proposal submission are deemed approved. For each position included in your subcontracting planprovide:
  1. Name of subcontractor;
  1. Primary contact name and email;
  1. Address of subcontractor;
  1. Description of services to be performed;
  1. Number of employees by job category assigned to this project; and
  1. Resumes (if appropriate and available).

Offeror Response

I-3.Training. If appropriate, indicate recommended training of agency personnel. Include agency personnel to be trained, the number to be trained, duration of the program, place of training, curricula, training materials to be used, number and frequency of sessions, and number and level of instructors.

Offeror Response

I-4.Financial Capability. Describe your company’s financial stability and economic capability to perform the contract requirements. The Commonwealth reserves the right to request additional information to evaluate an Offeror’s financial capability.

Offeror Response

I-5.Tasks. Describe in narrative form your technical plan for accomplishing the work using the task descriptions set forth below as your reference point. Modifications of the task descriptions are permitted; however, reasons for changes should be fully explained. Indicate the number of person hours allocated to each task. Include a Program Evaluation and Review Technique (PERT) or similar type display, time related, showing each event. If more than one approach is apparent, comment on why you chose this approach.

The following is an outline of the tasks required of the selected Offeror. In its proposal, the Offeror shall explain in detail how it will accomplish each of these tasks and provide all of the services required for a comprehensive licensing examination development and administration program.

  1. Examination Development and Preparation.
  1. The selected Offeror shall develop and prepare two separate examinations – the broker examination and the salesperson examination. These examinations are comprised of separate state and national portions. The Association of Real Estate License Law Officials (ARELLO) must certify the process of developing and weighting the questions for the national and state portions of the examinations. The content for the state portion of the examinations must be approved by the Commission. The examinations must be developed and prepared in accordance with recognized standards for test construction and evaluation.
  1. At a minimum, the following tasks are required for each examination:

a.The selected Offeror shall perform a job or task analysis. The selected Offeror shall identify the content of the examination by linking required job knowledge and skills to the examination’s specifications.

b.The selected Offeror shall employ or contract with a qualified psychometrician to assist with test development and evaluation.

c.The selected Offeror shall use subject matter experts and resources to write multiple-choice items that are related to the examination specifications. Subject matter experts shall be identified by the selected Offeror and approved by the Commission. Subject matter experts shall not be members of the Commission. Each item written shall be reviewed by the selected Offeror for subjective bias and readability and shall be analyzed statistically.

d.The selected Offeror shall field test examination items to determine how each item performs under actual examination conditions. The selected Offeror shall perform an item analysis based upon a review of the field-testing and rewrite examination items as appropriate.

e.All final items must be submitted to the Commission for review and approval.

f.The selected Offeror shall assemble approved examination items into the final examination.

g.The selected Offeror shall recommend a passing score, which shall be subject to the Commission’s approval. The passing score must distinguish minimally competent candidates. See Section I-5 D Examination Scoring for further requirements regarding scoring.

h.The selected Offeror shall prepare and maintain a current Examination Administration Manual that sets standards and procedures for ensuring uniform, secure examination administration.

i.The selected Offeror shall prepare and maintain a current Candidate Information Bulletin (CIB) that describes, among other topics, the application procedure and content areas of the examinations. See Section I-5 C.8 Examination Administration for further requirements regarding the CIB.

  1. Examination Revision
  1. The selected Offeror shall revise and amend the examinations to maintain their validity, reliability and legal defensibility.
  1. The selected Offeror shall perform examination item analyses as necessary using subject matter experts, at least once during the contract term.
  1. The selected Offeror shall review the item bank of each examination with the Commission in Harrisburg or another agreed-upon location, at least once, but not more than twice, annually. The review shall be scheduled by the selected Offeror with the Commission’s Administrator, and the selected Offeror shall pay any and all expenses related to the item bank review.
  1. The selected Offeror shall develop new examination items or revise existing items as necessary to reflect changes in the law and the Commission’s regulations. The selected Offeror shall continuously monitor any changes to the law and/or regulations. The selected Offeror shall revise and/or add examination items within three (3) months of the date on which the law or regulations were amended.
  1. All revised items must be statistically analyzed, psychometrically evaluated and approved by the Commission.
  1. Examination Administration
  1. The Offeror shall describe its examination administration policies and procedures, which shall include, but not be limited to, the following subjects: the handling of candidate complaints at examination locations, examination irregularities and emergencies, examination security, identity verification, the responsibilities of examination administration supervisors and other administration personnel and the cancellation and rescheduling of examination administrations in the event of inclement weather and similar emergencies.
  1. At a minimum, the selected Offeror shall administer each examination for which there is an eligible candidate on Monday through Saturday from 8:00 a.m. through 6:00 p.m. in Center City Philadelphia and three sites in contiguous counties to Philadelphia, Pittsburgh and two sites in counties contiguous to Allegheny County and in the metropolitan areas of Harrisburg, Erie, Allentown and Scranton/Wilkes-Barre.
  1. If candidate volume in any location results in a wait of more than two weeks for a candidate, additional locations within twenty-five miles of that location (or additional examination administrations) shall be established within thirty (30) days. Additional locations shall be approved by the Commission.
  1. The Offeror shall describe its policies and procedures regarding examination cancellation including the refund of examination fees to candidates and notification to candidates of cancellations.
  1. The selected Offeror shall provide the following information:
  1. To candidates – examination application forms, information necessary for all candidates to apply for all five categories of licensure and to obtain a criminal background check;
  2. To approved real estate education providers, colleges and universities – Uniform Real Estate Transcript Forms;
  3. To candidates, real estate education providers, colleges and universities and the Commission – current copies of CIB.

All distributed documents shall be the most current and be included in the CIB.

  1. The selected Offeror shall process examination applications (which includes the collection of examination fees) for all candidates.
  1. The selected Offeror shall prepare, print and distribute to their appropriate staff an Examination Eligibility Manual containing instructions for the evaluation of eligibility, a sample of which must be attached to the proposal. The selected Offeror shall review and revise the manual’s content as necessary. The selected Offeror shall post a copy of the manual on the secure website, as further described in Section I-5-C.24 Examination Administration.
  1. The selected Offeror shall determine the eligibility and provide notice of approval for the following examinations, in accordance with the applicable statutes and regulations:
  • Salespersons (see 63 P.S. Sections 455.521-522, 49 PA Code Section 35.272)
  • Rental Listing Referral Agents (see 63 P.S. Section 455.561, 49 PA Code Section 35.275)
  • Builder/owner Salespersons (see 63 P.S. Sections 455.551-552, 49 PA Code Section 35.274)
  1. The Commission determines eligibility for Brokers and Cemetery Brokers examinations.
  1. The selected Offeror shall prepare, print and distribute to the appropriate staff an Examination Administration Manual containing instructions for the secure and uniform administration of the examinations, including identity verification and examination security, a sample of which must be attached to this proposal. Fee information must be redacted from all samples of this RFP. The selected Offeror shall review and revise the manual’s content as necessary. The selected Offeror shall post a copy of the manual on the secure website, as further described in I-5-C.24 Examination Administration.
  1. The selected Offeror shall prepare, print and distribute a CIB that describes the methods of examination and scoring, examination administration procedures, the content areas of the examinations, procedures and fee for obtaining a duplicate score report, procedures for investigating and resolving alleged examination irregularities or suspected cheating and procedures for reporting and resolving candidate complaints. The CIB shall also contain forms that the selected Offeror requires candidates to use. The CIB and any updates thereto are subject to Commission approval prior to distribution.
  1. The selected Offeror shall provide a copy of the CIB to each applicant upon submission of an examination application and to any approved education provider, college or university within ten (10) days of a request.
  2. Upon request, the selected Offeror shall provide copies of the CIB to the Commission.
  3. When a revision to the CIB is made, a copy of the new version shall be provided to the Commission and the remaining paper copies of the prior version shall not be distributed and should be destroyed.
  4. The selected Offeror shall ensure that a sufficient supply of the current CIB is available at all times. (Requests may exceed the number of candidates.)
  5. The CIB shall also be posted on the selected Offeror’s website and shall be updated and posted within 30 days of the revision.
  1. All forms, applications, manuals, bulletins and other information provided by the selected Offeror must be up to date. The selected Offeror shall not distribute any written information regarding this examination program unless the information is current.
  1. The selected Offeror shall be available to provide customer service to candidates, at least, Monday through Friday, 8:00 AM to 6:00 PM Eastern Standard Time (EST). The Offeror shall describe in its proposal, the number of service representatives and proposed service level for responding to candidate inquiries, complaints and comments. At a minimum, the selected Offeror shall respond within 24 hours or the next business day.
  1. The selected Offeror shall establish, maintain and advertise a toll-free telephone number for use by Pennsylvania candidates. This number shall also be contained in the CIB.
  1. The selected Offeror shall schedule approved candidates for examinations within seven calendar days of receiving the application from the candidate. A candidate’s wait to take the examinations shall be no longer than two weeks from the date that the candidate is scheduled.
  1. The Offeror shall maintain adequate examination facilities and shall describe those facilities in detail in its proposal. At a minimum, each examination location:
  1. Must be safe and accessible to all candidates;
  1. Must have adequate lighting, ventilation, electrical supply, climate and other environmental features appropriate and conducive to the taking of examinations;
  1. Must be free of noise and other distractions;
  1. Must be legally accessible to individuals with disabilities;
  1. Must have adequate restroom and parking facilities;
  1. Must have on-site, limited-access security adequate to guarantee the integrity and confidentiality of the examinations and examination materials;
  1. Must have adequate seating and work space that is comfortable for adults and appropriate to the format of the examinations.

The selected Offeror’s facilities at each of the examination locations must, at a minimum, be reasonably comparable to one another.

  1. The selected Offeror shall prepare the examinations for administration, deliver the examinations to and from the examination sites and ensure the security of the examinations during all phases of preparation and delivery.
  1. The Offeror shall describe its procedure for admitting candidates to the facilities and verifying the identity of each candidate. At a minimum, the Offeror shall:
  1. Require the candidate to provide photo identification prior to being admitted into the examination location, and
  1. Photograph candidates at the examination location.

In the event that a candidate does not have photo identification because the candidate objects to being photographed for legitimate religious reasons, the Offeror shall implement a reasonable accommodation with the approval of the Commission.

  1. The selected Offeror shall permit authorized Department staff or other persons with written authorization from the Department to observe any examination at any time, without advance notice and upon confirmation of identity. No persons other than approved candidates, the selected Offeror’s examination personnel and Department staff or other persons authorized by the Department shall be admitted to an examination location.
  1. The Offeror shall employ a sufficient number of trained personnel, including proctors, to ensure professional and secure examination administrations. The Offeror shall:
  1. Describe in detail in its proposal the examination administration personnel/candidate ratio and list the qualifications and experience of examination facility supervisors.
  1. Provide, if requested, copies of the policy, procedure and instructional manuals used for training examination administration personnel and facility supervisors.
  1. Describe the plans to address proctor tardiness or absences (including last minute notification) to scheduled candidates and the Commission.