New FMCSA CSA 2010 Safety Measurement System

New FMCSA CSA 2010

Safety Measurement System

Orientation Material
12/8/2010[edited 1/20/2011]
TIA Orientation Material for the

New FMCSA CSA 2010 Safety Measurement System

TIA is pleased to present to our members, this orientation to the new Federal Motor Carrier Safety Administration (“FMCSA”) CSA 2010 safety measurement system, or SMS. This orientation material can be used by members to begin to understand how this first part of CSA 2010 works, and how its SMS is different from today’s measurement system, called SafeStat.

A Change is Imminent, But Not to Ratings

FMCSA has stated that it will replace the SafeStat measuring system, with its new SMS in December 2010. It is important to note that SafeStat and SMS are notrating systems. They are only measurements systems that provide guidance to FMCSA, to help FMCSA identify carriers for further review.

A later stage of CSA involves FMCSA’s intention to replace its current safety “rating” system for carriers. In order for FMCSA to replacetoday’ssafety ratings (known as “satisfactory”, “conditional”, and “unsatisfactory”), requires a formal federal rulemaking process, which may take as many as 18 months or more. Until that time, today’s safety ratings will remain unchanged.

Overview of the New Safety Measurement System (SMS)

The new SMS is very different than the SafeStat model, though they may appear similar at first. Under the SafeStat model, carriers were measured in four (4) very broadcategories called safety evaluation areas, or SEAs: 1) Driver; 2) Vehicle; 3) Safety Management, and 4) Accident. The Accident SEA has been hidden from the public for years. The data used to derive scores in these four categories included only: out-of-service violations, certain moving violations, and crash reports. Because such data is fairly narrow in scope, many carriers had no scores in the system, which left FMCSA wanting more information about more carriers.

The new SMS measures carriers in seven (7) more specificareas called Behavior Analysis Safety Improvement Categories (BASICs). These seven categories are: 1) Unsafe Driving (Parts 392 & 397); 2) Fatigue driving (Parts 392 & 395); 3) Driver Fitness (Parts 383 & 391); 4) Controlled Substances/Alcohol (Parts 382 &392); 5) Vehicle Maintenance (Parts 393 & 396); 6) Cargo-Related (Parts 392, 393, 397 & HM); and 7) Crash Indicator. The data used to derive the scores in the new system consist of all safety-based violations and all crashes, not just out-ofservice violations.The Crash Indicator and Cargo-Related BASICs will be hidden from the public.

SMS applies to American carriers. Canadian & Mexican carriers who cross the border into the U.S. receive a U.S. DOT number, and any U.S. inspections apply to their SMS data.

Please see Figure 1 below, for a comparative illustration of SafeStat’s 4 broad SEAs, versus CSA 2010’s SMS BASICs.

Figure 1: SafeStat SEAs vs. New SMS BASICs

Besides the significant change under SMSto the kinds of data thatare being collected to make a score, another significant change is that drivers and behaviors attributed to drivers are far more detailed under SMS. There are four (4) driver related BASICs under SMS, where under SafeStat, there was only one—the Driver SEA. This extra attention and/or scrutiny that SMS places on driver behaviors is one of the most important features of CSA 2010, and its new SMS. It will likely require motor carriers and their drivers to pay closer attention to multiple areas, or behaviors.

SMS BASICS and Examples of Violations

The seven BASICs reflect theFMCSA’s attempt to track and review data on behaviors, which the Agency believes contribute to truck safety. The BASICs and brief examples of the behaviors or violations are shown below, in Figure 2. BASICs with an asterisk (*) are those with the highest relationship to future crashes.

Seven BASICs / Examples of violations include:
Unsafe Driving BASIC (*) / Speeding, reckless driving, improper lane change, and inattention
Fatigued Driving (*) / HOS, ill, fatigued, logbook errors
Driver Fitness / Invalid/no CDL; endorsements, medical certificates
Controlled Substances & Alcohol / Use/possession alcohol, illegal drugs, misuse of Rx
Vehicle Maintenance / Brakes, lights, mechanical defects, failure to repair
Improper Loading/Cargo Securement / Shifting Loads, spilled or dropped cargo, unsafe HM
Crash Indicator (*) / Histories/patterns of high crash involvement

Figure 2 - BASICS and Examples of Violations

How Data is Collected

Data is collected from all roadsideinspections, violationsand accident reports. Nine states (CO, DE, GA, KS, MD, MN, MO, MT, NJ) have been part of the FMCSA’s SMS pilot program for many months, and are fully trained and their data gathering and uploading techniques are implemented, and their conversion to SMS will begin “seamlessly” when SMS begins. All other states have been training for the past several weeks. It is anticipated that their training will take through 2011 to complete, yet data will be gathered and uploaded at time of SMS’s launch. As with anything new, there will be a learning curve, and will likely be some errors or inconsistencies, most notably between the pilot states and non-pilot states. It should be noted, however, even when there was only SafeStat, and it was the only modelin the U.S., procedures and enforcement methods varied greatly between states.

Determining BASICs—Weighting, Exposure and Normalizing Collected Data

Once roadside data is gathered, the data is entered into the Motor Carrier Management Information System (MCMIS). Then, four steps are taken. They are as follows:

1. FMCSA must first associate the relevant data to an entity (defined as driver or carrier).
2. Next is to classify the data into: (a) the appropriate BASIC; (b) apply a “weight”to the data according to time (most recent = heavier) and severity; (c) normalize the data (e.g., type of carrier, number of trucks/vehicle miles traveled, or number of inspections).
3. FMCSA then compares each entity’s BASIC measure to other entities with a similar number of safety events
4. The fourth step is to assign a rank and percentile to the entity in the BASIC.

The severity weighting of violations is a key component of the new system. The severity weight of each violation is determined by its relative relationship to crash risk. Some BASICs like driver fitness and cargo-related do not have as high a correlation to crash risk as other BASICs, but they are critical to compliance with the Federal Motor Carrier Safety Regulations.

The severity weights range on a scale from 1 to 10, where 1 represents the lowest crash risk and 10 represents the highest crash risk. The FMCSA has indicated that the weighting of violations in each BASIC differs from one another, and cannot be compared across the various BASICs. The severity weighting of each violation was determined by the FMCSA by following a six-step process. The idea behind the process was to help form violation groupings and associated severity weights. The FMCSA has said that this process differentiates the SMS from SafeStat and addresses some of the criticisms of the SafeStat algorithm.[1]

Once violations and crashes have been placed into the proper BASIC and weighted based on time and severity, the scores are then adjusted or “normalized” to account for difference in exposure in entities. The normalization varies depending on what is being measured. Some BASICs are normalized based on number and type of truck, while others take into consideration the annual vehicle miles traveled to account for differing exposure risks.

Once the scores are normalized, the SMS then places carriers into “Safety Event Groups”. This tiered approach in placing carriers in “Safety Event Groups” accounts for the greater variability in violations and crash rates for smaller carriers based on small samples or limited levels of exposure.

After the measure for an entity has gone through these steps, then the entity is given a score or percentile ranking in each BASIC. This percentile ranking (from 1-100) shows the carrier's relative safety performance in that BASIC. The worse the carrier’s performance is relative to others in its peer group, the higher its score.

BASIC Percentiles & Intervention Thresholds

The FMCSA has created “intervention” thresholds (Figure 3, below), which differ by carrier type and basic to aid the agency in prioritizing interventions. The BASIC scores allow FMCSA to prioritize both of (a) which carriers receive an intervention, and (b) the type of intervention taken. Notice of a carrier’s intervention will likely not be visible to the public, however, violation data from investigations will be visible.

Interventions are actions taken by FMCSA against carriers, which may include any of the following: 1) Warning letter, 2) Targeted Roadside Inspections, 3) Off-Site Focused Review, 4) On-Site Focused Review, and 5) On-Site Comprehensive Review. Some interventions may result in a fine or penalty. Interventions will be discussed in more depth later in this document.

The intervention threshold is not a determination of whether the carrier is safe or unsafe, nor is it a safety rating, nor should the public imply that it is a safety rating. The fact is, until the safety rating rulemaking is complete (in about 18 months) BASIC scores have no impact on safety rating.

The intervention threshold representsrelative crash risk.[2] If a carrier has a percentile ranking above the pre-set intervention threshold set by the FMCSA, the carrier is then targeted for some type of intervention.[3]

Current FMCSA Intervention Thresholds by Carrier Type

BASIC / General / Haz-Mat / Passenger
Unsafe Driving* / ≥65% / 60% / 50%
Fatigued Driving* / 65% / 60% / 50%
Driver Fitness / 80% / 75% / 65%
Controlled Substances/Alcohol / 80% / 75% / 65%
Vehicle Maintenance / 80% / 75% / 65%
Cargo-Related / 80% / 75% / 65%
Crash Indicator / 65% / 60% / 50%

Figure 3 Thresholds set by FMCSA Policy by type of carrier

The Goal of SMS

The goal of the new SMS isto make the roads safer for everyone. The goal of the new SMS helps the FMCSA by identifying entities (defined as both carriers and drivers) needing interventions; determine the specific safety problems incurred by an entity; and monitor the safety issues throughout the intervention process – until the carrier or driver, through evidence gleaned from subsequent inspection or crash reports, can exit the intervention process or until FMCSA determines if further intervention/enforcement is necessary. TheFMCSA has stated that it believes that the new SMS will “empower carriers and other firms involved with the motor carrier industry to make safety-based business decisions.”[4]

Types of Interventions

The FMCSA has identified a number of interventions that it will use if a carrier has a score over the threshold in any BASIC. The agency will apply the specific intervention most likely to compel the carrier to take necessary remedial action. Depending upon the severity and number of deficiencies, the agency may skip some of the lower level interventions and move directly to an on-site review of the carrier.

The interventions are:

1)Warning letter

2)Targeted Roadside Inspections

3)Off-Site Focused Review

4)On-Site Focused Review

5) On-Site Comprehensive Review

The following is a brief description of the interventions:

1. Warning Letter – A warning letter will be sent to the carrier’s place of business. The letter will identify the deficient BASIC(s) and outlines the consequences if the safety issue is not addressed. The letter will provide instructions to the carrier on how to access its data so it can identify the source of the problem.
2. Targeted Roadside Inspections – The carrier’s trucks will be targeted in the inspection system for roadside inspections. The system will also provide roadside officers with detailed information regarding the carrier’s specific safety problems relative to the BASICs.
3. Off-Site Focused Review – The carrier will be directed to send records relating to the deficient BASIC. For example, if the carrier is deficient in driver fitness it may be required to send driver qualification files or specific driver documents to the enforcement agency. If the carrier does not supply the records it may be subject to an on-site review or other action.
4. On-Site Focused Review – The investigators will show up at the carrier’s place of business to review specific records related to deficient BASICs. An on-site review will most likely occur if the carrier is found to be deficient in two or fewer BASICs.
5. On-Site Comprehensive Review – An on-site comprehensive review is a complete review at the carrier’s place of business. This type of review is most similar to a Compliance Review. It will include all areas of a carrier’s operations, not just those that were over the threshold in a BASIC. This is most likely to occur if the carrier is deficient in multiple BASICs, has had a fatal crash, or has been the subject of a complaint.

Post Intervention Remedies and Follow-on Activities

Once an intervention has been made, there is also the possibility that the agency may take a number of follow-on activities as a result of the intervention. Most of these activities are similar to the enforcement actions the agency currently uses during an investigation. They are as follows:

Cooperative Safety Plan/ (NEW) – the Cooperative Safety Plan is a plan entered into “cooperatively” between the agency and the carrier. The idea of the plan is for the carrier to identify strategies and actions it will take to address safety problems discovered during an intervention.

Notice Of Violation – The Notice of Violation is a formal notice to the carrier of safety deficiencies. This “Notice” requires a response from the carrier. The agency uses this Notice as a way to take formal action where needed but where a civil penalty may not be appropriate. To avoid escalation of the Notice of Violation to a civil penalty, the carrier must produce evidence to the agency that it has taken corrective action.

Notice of Claim – The Notice of Claim is an enforcement action where the agency determines that the violations are serious and issues a civil penalty (usually a monetary fine) to the carrier.

Settlement Agreement – This is a negotiated agreement between the carrier and the agency. The carrier agrees to implement certain remedies to the identified safety deficiencies and in turn the agency defers or reduces the penalties.

Operations Suspended – A suspension order is an order for the carrier to cease operations. This is the most severe penalty and is issued to those carriers that have continuous and serious safety deficiencies.[5]

Many of the lower tier interventions are simply to notify the carrier of a potential problem and give the carrier an opportunity to take corrective action early, before the higher tier interventions are triggered. For this reason it is important to understand the various interventions and the likelihood of future enforcement action.

Carriers that correct an issue with a lower tier intervention may exit the system when their percentile ranking drops below the intervention threshold. These carriers will continue to be monitored by the FMCSA and their state partners but may not warrant any further action.

To find out more about the CSA 2010 program:

To obtain a copy of the new SMS methodology, go to:

1

[1]FMCSA- Safety Measurement System (SMS) Methodology Version 2.0, August 2010, pg. A-2 – A-3.

[2] “At the time of this publication, FMCSA has changed its public explanation for how thresholds were set. Previously the agency publicly stated thatallthe BASIC thresholds were set in policy, and in fact preliminary findings from the University of Michigan Transportation Institute (UMTRI) study had shown some of the BASICs did not have a strong correlation to crash risk. (Three of the BASICs – unsafe driving, fatigue and crash have shown a correlation to crash risk – however the UMTRI study was undertaken prior to the change in the SMS methodology which occurred in August 2010). In recent days, FMCSA changed its website and written explanation for setting thresholds from based on Agency “policy” and subject to change, which implied thresholds were set according to FMCSA resources and ability to manage the program. Now, FMCSA states that all BASICs are based on relation to relative crash risk. TIA has formally asked the Administrator and the CSA program manager to provide TIA with the study(ies) showing this correlation to crash risk on all the BASICs, under the current methodology. Until such time, TIA remains uncertain and skeptical about the origin and importance of the thresholds that have been set. TIA will update this sectionappropriatelywhen FMCSA responds with our request for data.”

[3] The full range of interventions can be found at:

[4]FMCSA- Safety Measurement System (SMS) Methodology Version 2.0, August 2010, pg. 1-2.

[5]The full range of interventions can be found at: