Exposure Draft September 8, 2017 Please use the review tools in Word to add proposed revisions and comments
Standard on Automated Valuation Models (AVMs)
Table of Contents
1. Scope
2. Introduction
2.1 Definition of an AVM
2.1.1 Preliminary Data AVM [AVM Assisting Appraisers]
2.1.2 Interactive Valuation Application AVM [Appraiser-Assisted]
2.1.3 Repetitive AVM [Continuous Application AVM]
2.1.4 Blended or Cascading AVM
2.1.5 Research AVM
2.2 Purpose of an AVM
2.3 Development and Application
2.3.1 Scope of Work
2.3.2 Identification and Acquisition of Property Data
2.3.3 Data Analysis Exploratory
2.3.4 Stratification
2.3.5 Data Representativeness
2.3.6 Model Specification
2.3.7 Model Calibration
2.3.8 Quality Assurance
2.3.9 Model Application and Value Review
3. Specification and Calibration of AVM Models
3.1 Data Quality Assurance
3.1.1 Data Availability
3.1.2 Data Verification
3.1.3 Qualitative and Quantitative data
3.1.4 Property Identification and Location
3.2 Model Specification and Calibration Methods
3.3 Stratification
3.4 Time Series Analysis
3.5 Calibration Techniques
3.6 Calibration Summary
3.7 Independent Variable Selection for Models
3.8 Location
4. Market Analysis and Intended use
4.1 Identify the Property Class/Type to be Valued
4.2 Identify Intended Use
4.3 Identify Comparable Sales or Substitute Economic Information for Sales Data
4.4 Identify the Valuation Approach(es) to be Used
4.5 Identify Property Characteristics that Have the Greatest Influence on Value
4.6 Identify Geolocational and Economic Influences
4.7 Cautions
5. Model Quality Assurance
5.1 Data Quality Assurance
5.2 Data Representativeness
5.3 Model Diagnostics
5.4 Ratio Studies
5.4.1 Measures of Accuracy Level
5.4.4 Price Related Vertical Inequities
5.4.5 Guidelines for Evaluation of Quality
5.4.6 Importance of Sample Size
5.4.7 Outliers
5.5 Holdout Samples
6. Documentation and Reports
6.1 Restricted Use Report
6.2 Appraiser-Assisted AVM (AAAVM) Report
6.3 Documentation Report
6.4 Uses of AVM Reports
6.5 Value Reconciliation
6.7 Mapping Report
7. Glossary
References
Suggested Reading
Appendixes
Appendix A. Model Specification
A.1 Cost Approach
A.2 Sales Comparison Approach
A.2.1. General Model
A.2.2 Direct Market Model Forms
Appendix B. Value JUSTIFICATION
Appendix C. Statistical Methods for Developing Location Adjustments
Appendix D. Time Series Analysis
Appendix E. Residential AVMs
E.1 Single-Family
E.1.1 Cost Models
E.1.2. Comparable Sales Models
E.1.3 Direct Market Models
E.2 Time Series Models for Residential Property
Appendix F. Valuation Models for Commercial and Industrial AVMs
F.1 Cost Models
F.2 Sales Comparison Models
F.3 Income Models
F.3.1 Modeling Gross Income
F.3.2 Vacancy and Collection Losses
F.3.3 Modeling Expenses
F.3.4 Direct Capitalization
F.3.6 Property Taxes
F.4 Quality Assurance
Appendix G: Land Models
G.1 Land Valuation Model Specification
G.1.1 Property Use
The analyst should estimate value based on the use of the property. The market will indicate in the location analysis will determine the highest and best use value and indicate where use is changing.
G.1.2 Location
G.1.3 Physical Characteristics and Site Influences
G.2 Land Data Collection
G.3 Development of the Model(s)
GI.3.1 Land Valuation Modeling by Sales Comparison
G.3.2 Land Valuation Modeling by Income
GI.3.3 GAMA Geostatistically Assisted Mass Appraisal
Appendix H: Statistical Tables
Table 1. Example of Ratio Study Statistical Analysis Data Analyzed
Table 2. Ratio Study Performance Standards
Appendix I: Uses of AVM Reports
I.1 Real Estate Lenders
I.2 Real Estate Professionals
I.3 Government
I.4 General Public
I.5 Ad Valorem Tax
1. Scope
This standardprovides guidance for public and private sector appraisal activities, single property appraisals,and appraisal review programs, real estate portfolios, and mortgage backed securities thatdepend on Automated Valuation Model (AVM) systems.The standard provides guidelines, principles,and best practices fordeveloping and using AVMs for the valuation of real property. AVMs can be usedwhen sufficient economic data exists to permit development of representative and valid statistical samples.In addition, theappendixes present the methodologies used in developing and implementingvarious types ofAVMs and related topics.
2. Introduction
2.1 Definition of an AVM
All automated valuation models (AVMs) should be developed by market analysts (appraisers/valuers), who use mathematically-based applications to perform analysis in the development of market value estimates by selecting the best statistically generated simulation of market activity by the analysis of location, market conditions, and property characteristics from information that was previously collected. AVMs are designed to generate value estimates of subject properties at specified points in time (retrospective or prospective dates).
The five types of AVMs based on intended use are:
- AVM Assisting Appraisers [Preliminary Data AVM]
- Appraiser-Assisted AVM [Interactive Valuation Application AVM]
- Repetitive AVM [Continuous Application AVM]
- Blended or Cascading AVM
- Research AVM
AVMs use:
- the sales comparison approach;
- cost approach;
- income approach
2.1.1 Preliminary Data AVM [AVM Assisting Appraisers]
After analysts develop the AVM application, appraisers use these AVMs in their professional assignments. The AVM sorts large amounts of electronic data and provides selected raw or basic data for interpretation by the appraiser. The appraisers use the AVM applications to provide their opinions of value in various standard valuation reports. These appraisers may provide their explanations and use their own data, which they have collected or properly verified.
2.1.2 Interactive Valuation Application AVM [Appraiser-Assisted]
This is a mathematical model application, or a set of applications, that are developed, calibrated and checked by analysts with valuation knowledge. In these model applications, the analysts use data that has been previously collected and verified. The data should be confirmed by the analyst through statistical methods. The models should generate output that is in compliance with relevant IAAO standards on mass valuationand national standards.
2.1.3 Repetitive AVM [Continuous Application AVM]
In this type of AVM, mathematical applications are prepared by an analyst after market analysis. The AVM application is intended to be used repeatedly to project values for future dates, without recalibration but through the addition of new sale prices and economic information. This process may reduce the reliability of the AVM application. If analysts are unsure of the valuation application’s ability to generate prospective valuations, the AVM application should be recalibrated.
2.1.4 Blended or Cascading AVM
the greatest number of properties to be evaluated by The results of multiple AVM applications are presented to the user within a single interface (value estimate). The cascade process allows the user to leverage the strengths of multiple AVMs by fitting for location, property type, and projected price range. Their weakness is in the ability for a user to manipulate the final estimates of value by keeping the process running while making changes to both the order of the process and the models included until the best results are returned. Most of the commercial products in current use are based on the cascading design thus allowing the client the ability to value the largest number of properties through a single interface. This also aids in the valuation of portfolios consisting of mortgage backed securities by increasing the number of hits during the run.
2.1.5 Research AVM
Research AVMs are a general purpose valuation tool which superficially resemble production AVMs in design, but have limited functionality. Research AVMs are used for initial testing of concepts and are only used for testing purposes. They find use in both academic research to measure trends in real estate values and in public administration as a tool for generating value estimates outside the of the normal assessment cycle.
2.2 Purpose of an AVM
The purpose of an AVM is to provide an accurate, uniform, equitable estimate of market value at a given point in time. AVM values that are reviewed for reliability and generated in compliance with regulations of the governing bodies (for example, The Appraisal Foundation, USPAP Standard 6) are considered appraisalsprovided the modeler follows the procedures for AVM development and any required reporting of results.Models that adhere to the best practices for data verification,data analysis,market analysis, and ongoing quality controlpresent the most reliable value estimates.
2.3Development and Application
The development of an AVM uses appraisal principles and techniques, in which data are acquired and analyzed to develop a sample based on valid sold properties to develop a marketvaluation model that can be applied to similar properties (sold or unsold) in the same market area. These may be either individual properties of interest or all properties that meet the requirements of the model.The two major components of valuation modeling are specification and calibration. The model specification process identifies property characteristics (variables) that measure supplyand demandand develops the proposed model structure. Model calibration is the process of deriving coefficientsforthe variables previously specified. Specification and calibration techniques vary with the purpose of the AVM, type of property, available data, and the experience and knowledge of the market analyst.The basic steps to develop an AVM are:
- Createa scope of work
- Identification and acquisition of property data
- Exploratory data analysis
- Stratification
- Determination of data representativeness
- Model specification
- Model calibration
- Quality assurance
- Model applicationand value review
Model development,specification,calibration, and quality assurance are iterative processes that are repeated until statistical diagnostics are satisfactory.
2.3.1 Scope of Work
The scope of work is created to define the type of property and geographic area in which the AVM will be applied, and the steps required to develop and implement the AVM.The AVM supporting documentation should state all assumptions, special limiting conditions, extraordinary assumptions, and hypothetical conditions.
A key assumption in many AVM applications concerns the assumed use of the property. Most real estate databases contain the current use of property.Therefore, most AVMs assume that the current use is the highest and best use.
2.3.2Identification and Acquisition of Property Data
Prior to developing an AVM, the property data needs to be identified and acquired.In some cases, the property data is acquired before the AVM process begins.In other cases, the market analyst can determine what property data is necessary.Identification of the required propertydata iscontrolled by the interaction of the property data and prices as developed in the model.Property data elements that have a relationship to value may dictate the data collection methods.
Data falls into three general categories: property data, locational data, and market data. Property data are composed of elements that representphysical attributes of the property.Locational data takes into consideration market demographics, traffic, land use policies, and other geographic factors. Market data include sales, income, and replacement cost information.Data can be acquired from multiplesources:
- Internal resources
- Government sources
- Third party sources
- Public (news)sources
- Client and customer
A primary assumption is that data, collected by physical site inspection or observed through various electronic systems,is considered reliable for use in the AVM algorithm.This assumption is dependent on performing adequate quality assurance.At this point data and market analysis is performed. Exploratory Data Analysis provides the quality assurance of the data. It is also assumed that checks and edits will be made as new data is introduced into the system to ensure compatibility with model specifications.
2.3.3 Data Analysis Exploratory
Data characteristic or attribute analyses include:
- Data quality review—consistency of data entry, outlier identification and removal;
- Data distributions—frequency of value-related characteristics;
- Market patterns—trends in property characteristics, locational analysis using sales data;
- Time trends—sales frequency and sales dates;
- Determination of reasonableness of data elements
- Determination of the age or collection dates of the data elements; and
- Determination of reasonableness ofdata sources.
2.3.4 Stratification
Property data can be stratified to improve model interpretation, quality review and results.Stratification may be done by type of property, location, or other property characteristics. (see 5.2 and 2.3.5 Data Representativeness)
In stratification, parcels are organized into homogeneous groups such as use, physical characteristics, or location. Properties are first stratified by use such as agricultural, apartments, commercial, industrial, and residential. Additional stratification by physical characteristics or value ranges may be performed to minimize the differences within strata and maximize differences among strata. Geographic stratification may be usefulwherever the value of various property attributes varies significantly among areas and is particularly effective when housing types and styles are relatively uniform within areas (IAAO 2011, 139–143). However, excessive stratification may provide too little variation in the data.It may be possible to create a global valuation model without stratification, if adjustments for strata attributes (location, use, age, style, etc.)are included as part of the model specification and calibration processes.
2.3.5 Data Representativeness
To produce quality results, data used in developing an AVM model shouldreflectthe subject property or population of properties and not over- or under-represent any subset of the population.The available sales may limit applicability of the AVM model. (see 5.2)
2.3.6 Model Specification
Model specification is the process of determining the format (mathematical form such as additive and nonlinear) of the AVM. The market analyst shoulddetermine the model to be employed and specify the variables to be used.
2.3.7 Model Calibration
Calibration is the process of determining the candidate variables to be included in the model and examining the coefficients in an AVM to insure they are logical. Several statistical tools can be used to calibrate AVM models (see Section 3Specification and Calibration of AVM Models). The methods of calibrating the various model types can be found in texts on modeling and simulation.
2.3.8 Quality Assurance
An AVM shouldbe tested to determine if it meets required accuracy and uniformity standards before being deployed and after deployment as needed, depending on the risk management policies. This is accomplished through statistical diagnostics and ratio studies in which value estimates are compared to actual values for the same properties. GIS can be usedto analyze the spatial trends of the value estimates.Before it is implemented, the AVM also should be tested using sale pricesthat were not used in the calibration process (e.g., holdout sample or other cross-validation techniques).Properties with unusually large residuals, atypical characteristics, or extreme ratios of model estimates to sale prices, termed “outliers," should be reviewed. Outliersare cases where it is likely that the sale prices (or other value serving as the dependent variable in the model) are not representative, the data are partially incorrect, or the property exhibits atypical features that cannot be adequately accounted for in the model. If the data cannot be corrected, the property should be removed from the sample. (See Section 5.4.7)
2.3.9Model Application and Value Review
Once tested and validated, the AVM application can be applied to properties of the same type in the area or region where the model applies. These values should be reviewed for reasonableness and consistency. It is also good practice to systematically review the generated values for reasonableness and consistency with similarproperties in the same marketarea.
3.Specificationand Calibration of AVM Models
In practice, specification and calibration are performed in an iterative process, which includes the following steps:
- Specify a model
- Calibrate the model
- Test the model
- Make adjustments to model specification
- Recalibrate the model
- Test the model
- Continue to repeat the process until statistically significant improvement is minimized
The AVM specification and calibration iterative process assumes that data are collected and verified in a consistent and professional manner(Standard on Verification and Adjustment of Sales 2010).
3.1 Data Quality Assurance
Data quality assurance includes evaluation of data availability and accuracy of all physical and market data including property identification and location(e.g., Exploratory Data Analysis).
3.1.1 Data Availability
The model specification process begins with an evaluation of the data availability. The availability of data will influence the specification of the model and may indicate the need for revisions in the specification and/or limit the usefulness of the resulting value estimates. Attributes used in the model should be examined for quality and completeness and to ensure that attributes are adequately represented. Publicly available data from sources, such as assessors and commercial sector third-party information services are the basis for most statistical models.As AVM use increases, commercial data brokers are emerging to service model developers and operators.These data warehouses aggregate large amounts of data from all available sources.When using second and third generation data, it is good practice to verify the accuracy of elements found in the data prior to inclusion in the model. The AVM market analyst shoulduse statistical data analysis,as described in Section 5 to confirm the assumption that the quality of the data will provide reasonable support for the modeling process.The developer should analyze the data to determine if it is sufficiently accurate for the intended use.AVM models are based on a sample of the universe of data. The specification process shouldreview the sample data used to develop the model as well as the population to which the model will be applied. The sample should be representative of the population (see Section 5.2) in all key elements of value including the types of properties, market conditions, value range, and land and building sizes, and building ages. Property types for which market information is not available should be excluded from both the sample and total population files as the model specification will not be representative of these properties.In addition, model results should not be applied to under-represented sections of the population.
Knowledge of key property characteristics is crucial to model specification. Limitations in the integrity and availability of the data affect the model specification. Models should be specified after a thorough understanding of the data in the sample and population.