2002 Peer Review
Measurement Program
Final Report
QUALITY ASSURANCE TEAM
HIGHWAY MAINTENANCE DIVISION
OFFICE OF MAINTENANCE
March 2003
INTERNAL USE ONLY
2002 Peer Review Report
Table of Contents
Section Page
Executive Summary 1
Major Program Findings………………………………………………………………………. 2
Scope 2
Methodology……………………………………………………………………………………. 3
Maintenance Category Highlights and Recommendations 6
Process Recommendations 12
Appendix 1 - Desired Maintenance Conditions…………………………………………… 13
Appendix 2 - Calculation of Level of Service 17
Appendix 3 - Element and Category Weights 19
Acknowledgements……………………………………………………………………………. 21
Executive Summary
This is a report of the 2002 Peer Review Measurement Program (PRMP). The report focuses on state highways that comprise the Interstate, Primary, and Secondary Systems maintained by the Maryland State Highway Administration (SHA), and is based on field evaluations conducted from June through October 2002.
Teams, comprised of maintenance personnel possessing a wide range of experience, and coming from different geographical areas, traveled to 28 different maintenance shops to review SHA maintained highways. These Peer Review teams spent two days in each shop’s area of responsibility to assess the condition of our highway system. In total, these teams reported on 1,790 randomly selected one-half mile roadway sections.
This report contains a statewide assessment of our highway system based on the 2002 and previous years’ findings. Information at the shop and district levels will be available for the managers of the specific areas detailing information on each maintenance element reviewed in the program.
The results of the 2002 program indicate that the overall quality of maintenance on state highways is excellent. While the overall maintenance of our highway system is excellent, many areas are clearly in need of attention. In some cases, SHA as a whole or a shop in particular may need to change its priorities to address shortcomings.
A major benefit of past Peer Review reports was the ability to document differences in the findings on a year-to-year basis. The availability of five consecutive years of data enables managers to note trends in levels of service. Each unit of responsibility (state, district, and shop) can measure its own progress from year to year.
Closer scrutiny of maintenance elements has given managers a clearer picture of the system at all levels. The Quality Assurance Team (QAT) looks forward to the day when Peer Review data could be used by SHA shops as a tool to help prioritize areas of need, as well as to determine levels of funding, personnel, and equipment.
Major Program Findings
· The statewide highway maintenance level of service (LOS) for 2002 was 90.2% on a scale from 0 to 100, showing little change from the 2001 LOS of 90.3%. In addition, the 2002 LOS is exactly the same as the average LOS of the previous four years. While the overall level of service remains consistent, each major category and many individual elements have changed (up and down) from year to year.
· Statewide levels of service for each of the five maintenance categories and the overall condition are as follows:
Category / 2002 LOS / 2001 LOS / 1998-2000 Average LOSTraveled Roadway / 87.0% / 89.7% / 88.7%
Shoulders / 90.2% / 91.1% / 89.4%
Drainage / 93.6% / 90.4% / 92.0%
Traffic Control/Safety / 93.9% / 92.8% / 92.5%
Roadside / 84.5% / 86.0% / 87.8%
Overall / 90.2% / 90.3% / 90.2%
· In 2002, the LOS for individual maintenance elements ranged from a low of 57.3 for flexible roadway cracking to a high of 100% for rigid roadway faulting and spalling. All in all, 32 of the 39 maintenance elements reviewed had a LOS of 80% or greater.
· A detailed review of each of the elements and team recommendations may be found elsewhere in this report.
Scope
This report is a study of the 5,208 centerline miles of roadway comprising the Interstate, Primary and Secondary Highway Systems maintained by the State Highway Administration. The program evaluated the level of service provided to our customers on randomly selected sections of highways in Maryland's seven engineering districts, consisting of 23 counties (28 maintenance shops). The review occurred during the summer and fall of 2002.
Methodology
The Peer Review Measurement Program is designed to provide SHA managers with an accurate, reliable evaluation of the condition of SHA's highways and roadsides, and the level of service we are providing our customers. It is critical we maintain our managers' confidence in the accuracy of the data collected in this program.
The Peer Review Measurement Program consists of teams, comprised of 3 people, reviewing the condition of highways maintained by each of SHA’s 28 shops. The 3 person teams are drawn from a pool of 14 people, seven from the Office of Maintenance (OOM) and seven from the districts. Using a limited pool of evaluators helps bring consistency to the program. OOM’s Quality Assurance Team sets up the schedule of shop evaluations. Every effort is made to combine the strengths of the various team members to assure we have quality teams for each review.
In addition to each 3-person team, the host shop provides a vehicle and a driver. The shop may also designate a "host representative" to serve as navigator and expert on local conditions in their county. Although each shop had the opportunity to designate anyone as the “host representative,” the RME and ARME are not encouraged to be present during the reviews.
The teams spend two days evaluating highways in each shop's area of responsibility. The teams evaluate approximately 65 sites, each ½ mile in length, during each review. In an earlier condition assessment program, the Peer Review for Quality Highway Maintenance Program, the number of sections evaluated in each county averaged 100, in some cases as many as 165. By reducing the size of the sample in 1998, the confidence level in each county's data may have been "statistically" affected. On the other hand, the accuracy gained by looking more closely at fewer sites should have led to greater confidence in the data. Still, the size of the statewide sample is adequate to maintain the 95% confidence level.
Each site is reviewed in detail while driving the shoulder (if one exists) or as slow as traffic permits. The teams exit their vehicle and walk the site when necessary. In addition, the teams pause after each site, discuss their impressions, and reach consensus about the condition of each maintenance element. One team member fills out evaluation forms, supplied by OOM, for the group. New for 2002 the team decided to walk one randomly selected ½ mile section within each management area to determine if our windshield evaluation were accurate. The overall findings of this check indicated that windshield evaluations are reasonably accurate with the only minor differences being noted in litter, ditches and culvert pipes.
The safety of the participants is a high priority of all Peer Review teams. Every precaution is taken to reduce the risk faced while driving and walking on the selected sections of highways. Participants wear safety vests when out of the vehicle. The vehicles are well marked and have warning lights. If it is not safe to slow down to observe the conditions on a roadway section, the team observes the section "on the fly" (at full speed). It is better to miss some data and reduce risk, than to compromise safety for accuracy. Participants are aware of everyone's safety. This year we required that only yellow state vans with safety light bars be provided for the tours.
During the reviews, the teams evaluate as many as 39 different maintenance elements in five categories: Traveled Roadway, Shoulders, Drainage, Traffic Control and Safety, and Roadside. The teams compare “actual maintenance conditions” against “Desired Maintenance Conditions (DMC)” on a "does meet/doesn't meet" basis. A Quality Breakthrough Team composed of field personnel from all areas of the state originally produced the DMC. The DMC is reviewed prior to the beginning of the season, and as needed during the season, by field personnel who are the experts on what is possible in maintenance. This yearly pre-season review of the elements provides training for new evaluators and helps to standardize our ratings.
An OOM participant records the team's findings onto Peer Review data forms, along with any comments made by the team. Elements meeting the DMC have a check placed in the "YES" column, while elements that do not meet the DMC have a check placed in the "NO" column. Elements not present in the section or not fully visible are marked with a "0" in either column.
Following completion of the statewide field surveys, data from each of the sections is entered into a database program. The data is validated and levels of service are calculated for each shop, district, and the state.
Levels of service (LOS) are based on the percentage of elements that meet or exceed desired maintenance conditions. This percentage has been calculated for each element. Individual elements are weighted within each “category” (which consist of groups of common elements) in order to calculate category LOS. Each category is weighted to obtain an overall LOS for each roadway section evaluated. The statewide LOS is calculated based on by the percentage of road miles within a system or a county. Examples of the calculations and category/element weightings can be found in Appendixes 2 and 3.
All references to LOS are based on a 0-100% scale, with a 100% LOS indicating that all desired maintenance conditions were met. Theoretically, anything from a newly constructed roadway to a well-maintained aged roadway can receive a maximum of 100% LOS on the scale. Levels of service ratings are as follows: an LOS of 90-100 is considered excellent, a LOS of 80-89.9 is considered good, a LOS of 70-79.9 is considered fair, and a LOS of 60-69.9 is considered poor.
Several additional notes need mentioning at this point. OOM's Quality Assurance Team makes every effort to add quality control to the program. They schedule Peer Review Measurement Program Team meetings prior to and during the season to assure that the evaluators are looking at maintenance elements in a consistent manner. The bottom line is we need to have accurate data that earns the confidence of maintenance managers, and could prove useful in determining the needs of the organization.
Traveled Roadway Category
Highlights and Recommendations
Flexible Roadway Levels of Service
Year / Cracking / Surface / Raveling / Rutting / Shoving / Bleeding / Flushing / Potholes / Total LOS1998 / 61.5% / 93.7% / 89.7% / 98.1% / 95.8% / 86.9%
1999 / 66.6% / 98.0% / 91.3% / 98.9% / 95.5% / 89.2%
2000 / 68.2% / 97.0% / 94.6% / 99.7% / 96.0% / 90.3%
2001 / 64.8% / 97.1% / 93.7% / 99.3% / 97.7% / 89.7%
2002 / 57.3% / 96.5% / 89.7% / 99.4% / 97.1% / 87.0%
Rigid Roadway Levels of Service
Year / Potholes / Cracking / Spalling / Joint Material / Faulting / Total LOS1998 / 82.9% / 74.7% / 64.2% / 62.2% / 91.7% / 75.9%
1999 / 88.8% / 92.0% / 95.1% / 96.3% / 100.0% / 94.2%
2000 / 91.0% / 88.1% / 92.9% / 98.2% / 100.0% / 93.7%
2001 / 92.9% / 95.6% / 97.1% / 90.3% / 100.0% / 95.2%
2002 / 82.7% / 76.7% / 100.0% / 95.8% / 100.0% / 76.8%
Rigid (Portland cement concrete) pavement elements range from 76.7% to 100.0%. Overall, the rigid pavement level of service fell from 95.2% to 76.8%
The drastic fall in this category may be attributed to the limited number of rigid pavement sites reviewed in the program. The sample size may be too small to allow for any conclusions. Also, many rigid pavements are repaired and overlaid with Hot Mix Asphalt without new sections of rigid pavement coming online.
Shoulders Category
Highlights and Recommendations
Shoulder Levels of Service
Year / Shlder Joint Sep / Surface/ Edge Raveling / Non-positive Drainage / Vege-tation Growth / Potholes / Cracking / Distortion / Drop-off /Buildup / Total LOS1998 / 86.0% / N/A / 99.0% / 85.2% / 96.6% / 74.9% / 94.7% / 81.2% / 88.0%
1999 / 85.9% / 94.8% / 98.0% / 87.4% / 96.9% / 82.0% / 97.9% / 88.2% / 91.2%
2000 / 86.9% / 93.0% / 97.2% / 89.4% / 96.4% / 80.0% / 97.2% / 87.4% / 90.7%
2001 / 84.5% / 93.4% / 98.7% / 91.2% / 98.2% / 80.7% / 98.8% / 84.5% / 91.1%
2002 / 82.0% / 92.8% / 98.2% / 90.6% / 97.7% / 76.9% / 98.6% / 87.3% / 90.2%
The statewide Shoulders Category slipped back in 2002. “Cracking” had a percentage of 76.9, down from the 2001 figure of 80.7%. This could be attributed to the aggressive crack sealing program we experienced from 1998 on is nearing the end of the crack sealants service life, and could need to be re-sealed. This also accounts for the LOS drop in the “Shoulder Joint Separation” and “Vegetation Growth” elements.
“Shoulder edge drop-off/buildup” improved from 84.5 percent to 87.3
The Peer Review Team recommends SHA place greater emphasis on crack sealing to address this concern.
Drainage Category
Highlights and Recommendation
Drainage Level of Service
Year / Ditches / Culverts / Catch Basins/ Inlets / Curb / Gutter / Total LOS1998 / 93.0% / 89.1% / 93.1% / 78.0% / 88.5%
1999 / 98.8% / 94.5% / 94.3% / 87.6% / 94.0%
2000 / 94.0% / 88.4% / 94.2% / 84.4% / 90.3%
2001 / 92.6% / 84.3% / 93.8% / 89.1% / 89.8%
2002 / 95.1% / 90.9% / 95.8% / 90.6% / 93.1%
The statewide Drainage Category showed an increase in all elements with a “Total Level Of Service” increase from 89.8% in 2001 to 93.1% in 2002. The rise in the level of service for this category may be attributed to the increased funding for overlays and streetscape projects over the past several years. It may also be attributed to a lesser extent to a change in the way the element is reviewed. Prior to 1999, excessive vegetation growth would cause the “curb and gutter” element to fail to meet the desired level of service. Since 1999, a separate element was created to capture vegetation growth. The “curb and gutter” element now only reflects its structural characteristics.