The Validity and Predictive Utility

of the

Banks & Weitzul

Behavior Profile Salesperson Selection System

(BPS)

A Report to:

The General Agents and Managers Association

A. Zinsser Consultancy

October 1996

Executive Summary

The General Agents and Managers Association (GAMA) sponsored an 18-month study to determine the predictive validity of the Behavior Profile Salesperson Selection System(BPS) developed by Banks & Weitzul as a supplemental measure to the Career Profile (LIMRA) among a sample of new and recent insurance agents. The BPS is a self-inventory personality test consisting of behavioral statements designed to assess temperament and other traits that have been found to be predictive of sales performance. The research design involved the collection of BPS and Career Profile (CP) scores, along with agent production and retention data among both new hires and a sample of agents already on the job six-months or less.

A total of 53 agencies across two companies participated in the study and provided sufficient data on 209 agents. Approximately 85% of the study sample were agents representing Northwestern Mutual Life. Significant moderate to high positive correlations were found between BPS scores and first-year commissions, first-year premiums, and the total number of life policies sold during the first year of employment. A significant relationship was also found between the BPS and the total number of disability policies sold among a sub-sample of 138 agents in the study who reported selling such policies during their first year. The Career Profile (CP) score also predicted first-year commissions and premiums earned. However, the strength of association between the CP test and these two criteria were of considerably smaller magnitude when compared to the BPS. The results also indicated that the BPS was predictive of agent retention, whereas the CP did not predict survival among agents in this study sample.

To assess the relative contributions of the BPS and CP scores to overall production, multiple regression analyses were performed. With respect to the prediction of first-year commissions and premiums, the results indicate that while the two measures together made a contribution of sizeable magnitude, the BPS provided the major contribution. High scoring agents earned twice the commissions and premiums when compared with low scoring agents using the BPS. CP scores also predicted differences in production, but the differences found were not as great. Based upon these findings, agencies may wish to consider use of the BPS as part of their agent selection program.

Table of Contents

Page

I.Background and Purpose...... 1

II.Description of Study Measures

The Behavior Profile Salesperson Selection System (BPS)...... 3

The Career Profile System (LIMRA)...... 6

III.Study Methodology

Validation Study Design...... 6

Study Participants...... 7

Measures of Agent Productivity and Retention...... 8

Data Collection Procedures...... 8

IV.Results

Prediction of Agent Sales Performance...... 8

Combining the BPS and CP to Predict Agent Productivity...... 13

Prediction of Agent Retention...... 13

Comparative Benefit of Selection Systems-Predicting the

Probability of Success...... 14

V. Practical Implications of Findings and Recommendations...... 17

References...... 19

List of Tables and Figures

Page

Table1Means, Standard Deviations, and Correlations Among Study

Variables:The Prediction of Agent Sales Performance...... 10

Table2The BPS and Career Profile as Predictors of First-Year

Agent SalesPerformance:Corrected Correlations...... 11

Table3The Prediction of Agent Survival/Retention...... 14

Table4Expectancy Table: Percentage of Agents at Three Different

Levels of Total Fist-year Commission by BPS and CP Score Ranges...... 15

Figure 1BPS Score by First-Year Total Commissions...... 12

Figure 2CP Score by First-Year Total Commissions...... 12

Figure 3Average Total First-Year Commissions Among Low, Medium,

and High Scoring Agents on the BPS and CP...... 16

iii

I.Background and Purpose

The General Agents and Managers Association (GAMA) is an organization with a main client base of insurance agency managers. These individuals run agencies that sell the insurance products of primarily one company. A significant factor in determining an agency or company's profitability rests with agent retention and productivity. After peaking at 20 percent in 1987, the Life Insurance industry's four-year retention rate dropped to 18 percent in 1993, reflecting the historical norms of the past two decades. At the same time, GAMA members are increasingly finding that management compensation schemes are predicated upon improving agent retention and productivity, giving field managers a vested financial interest in assuring that they have the best tools available to select agents.

As part of the Career Agent Tracking Study being conducted by The Life Insurance Management Association (LIMRA), one of the distinguishing characteristics of high retention agencies is that they use multiple selection steps as part of the agent hiring process. While the standard procedure for hiring sales agents within the industry is to screen applicants with a biodata or life history test, such as the Career Profile (LIMRA), and then conduct a follow-up interview among those above a certain minimum score, agency managers have voiced considerable interest in obtaining additional assessment tools in order to improve the effectiveness of their hiring decisions.

During 1995, GAMA learned of the Behavior Profile Salesperson Selection System (BPS) developed by Banks & Weitzul (Princeton, New Jersey), a test that they believed could provide added value as a supplement to the Career Profile as part of a multiple assessment screening program. The BPS has also been identified as a useful instrument for selecting insurance sales personnel independent of its use with other measures (Weitzul, 1987). This instrument assesses a candidates' temperament and derives behavioral trait and style information that can be used by agency hiring managers to predict insurance sales production and agent survivorship. GAMA also believes that a

further contribution of the BPS test is its potential value in identifying the management styles that would best develop individual agent performance given the type of marketplace.

GAMA's decision to bring the BPS to the attention of its members was predicated on the conduct and findings obtained from further independent research that would be sponsored by GAMA. To this end, GAMA contracted with A. Zinsser Consultancy to conduct an 18-month pilot study, with a limited but geographically diverse number of agencies representing 2-3 companies in the industry, in order to evaluate the predictive validity and utility of the BPS as a supplemental measure to the Career Profile using a population of new and recent hirees working as sales associates whose products are primarily whole life and term insurance.

The potential benefits to GAMA and its members in the conduct of this research study were considered to be:

1) the opportunity to evaluate and, if appropriate, introduce a valid and reliable psychological temperament instrument specifically developed for the insurance industry, based upon member interest in exploring the potential of using personality tests to predict the complex area of sales performance;

2) the opportunity to improve agent selection by augmenting the Career Profile with the BPS in order to improve retention and productivity; and,

3) the potential of offering agency members a vehicle that can improve the performance of those hired by providing managers with information and guidelines from which to plan an individual's orientation and development on the job.

Part II below provides a description of the BPS, including a summary of some of the available validation data concerning this test.[1] The Career Profile (CP) is only briefly described. Part III presents the study methodology used by A. Zinsser Consultancy to evaluate the BPS. This is followed by Parts IV and V of the report which present the results from the study and includes a discussion of the practical implications of the finding.

II.Description of Study Measures

Behavior Profile Salesperson Selection System

The Behavior Profile Salesperson Selection System (Banks & Weitzul) consists of two components, the 320 Questionnaire and the 120 Questionnaire. Each of these measures is described below:

Behavior Profile- 320 Questionnaire- This instrument, which takes about one hour to complete, consists of 320 questions presented in the form of descriptive behavioral statements that require a person to respond either "yes" or "no". That is, either agreeing or disagreeing with how a person sees himself or herself relative to the descriptive statement. This instrument was designed as a direct measure of temperament, but can be more broadly classified as a self-inventory personality measure.

According to Banks & Weitzul, the 320 Questionnaire was designed to measure the following seven primary traits or factors: overachiever; entrepreneur; active; passive; aggressive; sensitized; and, compulsive. These seven traits are supported by 31

sub-traits that are linked to each factor to provide for further meaning. The number

of behavioral statements per primary trait varies from 35 to 57 questions with no double scoring. A persons "score" for each primary trait is placed on a nine-point scale with different weighted values for each trait. A composite overall score is then calculated and converted to a scale from one to five (1-5) which defines the probability of success in sales.

Behavior Profile- 120 Questionnaire- This biographical questionnaire takes approximately 90 minutes to complete and asks the person to list his/her education, work history, preferred work positions, bosses, and work environments. It also incorporates a section where the person is asked to prioritize various life goals, and a concluding exercise requiring responses to incomplete sentences. The 120 instrument is used as a check of the findings obtained from the temperament measure and is incorporated into the final ranking decision. An empirically-keyed version of the 120 Questionnaire is in development, but could not be included as part of the current study.

Prior Validation Information on the BPS

To date, the validation efforts have been focused on the Behavior Profile- 320 (temperament) Questionnaire. A review of technical reports provided by Banks and Weitzul to A. Zinsser Consultancy describe the use of a number of strategies and research efforts designed to support inferences that: 1) the behavior traits being assessed by the BPS are both observable and measurable traits, and that various combinations of these traits can be used to form a trait pattern that define a persons "behavior style;" and 2), the behavioral trait and behavioral style information could be used to predict success in a business activity such a sales.

Content/Construct Validity

The first inference deals with the content/construct validity of the BPS. The available evidence consists of a content validation study that first involved the categorization of specific behaviors that were considered indicative of the relevant behavioral traits, followed by a comparison of the trait information obtained from the temperament questionnaire with trait information obtained through independent psychological interviews. With respect to any construct validity evidence, available information is limited to a discussion of the use of various factor-analytic procedures performed on individual items and sub-traits that comprise the BPS in order to identify the structure of the instrument and whether there is a statistical foundation to support the rational grouping of items into trait measures. No data are presented comparing the BPS with other personality instruments designed to measure similar traits as is found in many construct validation studies. The BPS does have a set of score correction procedures used to control for the influences of social desirability, which is typically a common general factor found in self-inventories that measure personality traits (Edwards, 1957). However, no separate score of social desirability is provided to the user. Concerns about how scores would be affected by situational factors, such as taking the test when applying for a job as compared to a period somewhat later when performing a job satisfactorily, appear to be of little issue given the acceptable six- and twelve-month average test-retest reliability coefficients of .84 and .80 found across the 31 sub-trait measures.

Criterion-related Validity

A technical report titled "Statistical Evaluation-Behavior Profile Salesperson Selection System" (Banks & Weitzul, 1987) presents data regarding the inferences that can be made between the BPS and such criteria as insurance sales production and agent survival among new and inexperienced insurance sales personnel. The basic findings from this research involving several thousand agents conducted over a three-year period indicated that the BPS can differentiate between high producers and low producers of insurance sales products, and that those agents expected to perform better by the BPS were found to have higher sales success when compared to a control group of agents hired into an insurance sales program without being tested with the BPS. Also, those agents predicted to be highly successful on the BPS, were more likely to have longer duration of survivorship when contrasted with those agents predicted to have either a low or moderate likelihood of success on the BPS.

The Career Profile System (LIMRA)

The Career Profile (CP) is a pre-employment biodata screening instrument developed by LIMRA International for selecting insurance agents. The CP ( also referred to as the ICP - Initial Career Profile) is an empirically keyed questionnaire that consists of 162 questions appropriate for use in evaluating the probability of success in an insurance sales career among individuals with no previous insurance sales experience. The validity of the CP has been well documented and revalidated continuously over fifty years of use in the Insurance industry (Brown, 1978; Thayer, 1977). The Career Profile Rating has been reported to be predictive of both agent productivity and retention (Kelly and McManus, 1994).

The CP Rating has been reported to measure several dimensions such as financial stability, income expectations, work history, career expectations, commitment, and social contacts (Crosby, Dalessio, & McManus, 1990). The CP test scores range can range from 1-19, but companies who use the CP as an early screen in the hiring process for new agents usually eliminate from further consideration candidates below a certain company specific cutoff score (e.g. 10 or lower).

III.Study Methodology

Validation Study Design

GAMA requested that A. Zinsser Consultancy design and conduct a criterion-related validation research project to determine the predictive validity of the Banks & Weitzul test (BPS), both independently and as a supplemental measure to the LIMRA Career Profile, using a population of new and recent hirees working as sales associates whose products are primarily whole life and term insurance.

The study design involved two periods of about nine months each focusing on different activities. During the first period predictor information in the form of CP and BPS test scores was collected on new or recently hired agents. The second period involved collecting criterion information on the level of sales performance achieved by these same agents over their first year. To complete the validation study with a sufficient number of study participants and within the 18-month pilot project period, the study incorporated both predictive and concurrent validation approaches. The concurrent approach involved the collection of Career Profile and BPS predictor scores, along with production data, among a sample of individuals already on the job six months or less. Sales performance measures were collected in three-month periods covering the agent's first year of performance with a particular agency.

Study Participants

GAMA solicited participation from six companies and 93 member agencies in order to obtain a study sample of approximately 200-300 agents from a mix of different agencies across two or three companies within the industry. Agency managers were asked to select those agents that were either new or recent hires who had no prior experience in selling insurance. These agents chosen also had to be responsible for primarily selling individual life insurance products.

A total of 53 agencies across two companies participated in the study (provided both predictor and production data on agents.) These agencies were geographically diverse and varied with respect to the size of their agent pools and the degree to which they used multiple assessment methods to screen new agents. While an initial sample of 280 agents agreed to participate, after attrition, the final group of study participants consisted of 209 agents. The majority of the attrition was due to agents who either left their agency shortly after hire, or had not completed their first year of service. Nearly 85% of the study sample consisted of agents from Northwestern Mutual Life and 15% were agents of New York Life.

Measures of Agent Productivity and Retention

Several objective measures of job performance were collected for the purpose of this study. The agent productivity measures consisted of first-year commissions, total dollar amount of premiums, total number of life insurance products sold; and, total number of disability products, for those agents who reported selling these policies during their first year . Agencies provided this information also on a quarterly basis covering the first 12 months of an agent's employment with the agency.

The data collected on retention consisted of determining whether the agent remained with the agency for at least 12 months after the date of contract. For agents no longer employed by the agency, agencies indicated whether the separation from employment was due to either voluntary departure or termination.

Data Collection Procedures

Agencies participating in the study received detailed written instructions for administering the BPS, as well as instructions for collecting and reporting other agent information such as Career Profile data, agent production results, and survivorship information. Written instructions were also provided for agents on completing the Behavioral Profile Questionnaires. Agents were asked to complete the two questionnaires as part of a project on improving agent selection methods and were informed that their responses would be kept confidential and used for research purposes only. Company/agency and agent specific identification numbers were used to capture and match all test and productivity data and to insure test confidentiality.