Certification Requirements for the Level 1 AIE Analyst

Certification Requirements for the Level 1 AIE Analyst

Certification Requirements for the Level 1 AIE Analyst

Overview

Hubbard Decision Research promotes the use of Applied Information Economics by qualified individuals. The AIE certification ensures organizations that use AIE that the certified individual is, in fact, proficient with the AIE method. There are four levels of certification in AIE each proving an increasing proficiency with the AIE method. This document explains the detailed requirements to pass both the written test and the PEC so that candidates can prepare themselves. A Level 1 Analyst is defined as follows:

“A level 1 analyst is qualified to assist on risk/return analysis project and AIE implementation projects. The level-1 analyst can build decision models, run Monte Carlo simulations, compute information values, and execute empirical studies. They understand various financial analysis techniques and sampling methodologies. A level-1 analyst works under the supervision of a level 2 analyst. Certification requires completion of level-1 training, a score of 85% on the level-1 Written Exam covering the AIE process and financial and statistical methods, and completion of the level-1 Practical Experience Checklist. “

In other words, it is the objective of the Level 1 Analyst certification to ensure that an individual is capable of performing all the tasks described above. The material required to prepare for the written test are PEC are:

  1. Attendance of the AIE training seminar (in rare cases, Level 1 training may be waved subject to the judgment of the assigned evaluator)
  2. Reviewing printed training material provided in their Level 1 training
  3. The book “How to Measure Anything: Finding the Value of Intangibles in Business”
  4. Practice with the AIE tools, especially the AIE Monte Carlo tool with VIA calculations
  5. Optional: The candidate may review any introductory statistics or finance text if they feel they are too unfamiliar with these basic concepts

Written Test

During the proctored written test, the candidate will be expected to perform each of the following tasks. The candidate will need a score of 85% or better to pass this portion of certification.

  1. Demonstrate proficiency with basic cost/benefit modeling including:
  2. Building one or more decision models based on written case explanations and oral interviews of the proctor
  3. Explaining how to elicit definitions of intangibles from clients by using the clarification chain, thought experiment or other methods of clarifying measurement definitions
  4. Identify common exclusions and error in business cases
  5. Explaining and using basic financial calculations like NPV and IRR
  6. Demonstrate proficiency with calibration concepts including:
  7. Explaining the calibration process for both ranges and binary questions
  8. Computing and compare expected vs. actual percent correct
  9. Explaining methods for improving calibrated estimates including the equivalent bet and other methods
  10. How to address certain common questions or objections about calibration
  11. Using the Excel calibration score template
  12. Demonstrate proficiency with modeling uncertainty by:
  13. Calculating simple probabilities of events given other simple probabilities and conditions
  14. Explaining differences among the five distribution types used in the AIE tool and situations where they would each be used or potential issues with their use in certain situations
  15. Conducting and interpret the Monte Carlo simulation of a decision in an example
  16. Using the decision model and Monte Carlo results to identify potential risk mitigation and value enhancement strategies
  17. Eliciting an investment boundary from a client (portrayed by test proctor) using the AIE investment boundary template
  18. Set up a simple conditional probability simulation in a Monte Carlo
  19. Demonstrate proficiency with information value calculations including:
  20. Explaining opportunity loss and expected opportunity loss
  21. Computing and explaining information values for basic binary decision problems
  22. Computing information values with the AIE tools and interpret the results including the explanation of each of the columns in the tool
  23. Computing a marginal EVPI
  24. Demonstrate proficiency with measurement methods
  25. Given an EVPI and threshold of two or more particular uncertain variables, identify and explain potential measurement methods including
  26. Decomposition
  27. Instinctive Bayesian
  28. Non-Bayesian empirical methods (see list in 5.b below)
  29. Bayesian methods (see list in 5.b below)
  30. Given AIE empirical methods templates, explain and use empirical methods including:
  31. Computing a 90% CI of the mean of a population with small or large samples
  32. Computing a 90% CI of the proportion of a population having some property based on a sample of the population
  33. Use of the method for “measuring to a threshold”
  34. Computing the 90% CI of the difference between a test group and control group in a controlled experiment
  35. Use of a simple (not a range) Bayesian inversion for binary probabilities
  36. Demonstrate familiarity the components of an AIE risk/return analysis project including:
  37. Explaining the basic AIE philosophy and approach to measurement
  38. Explain the roles in a project and a workshop
  39. Explain the types of workshops
  40. Explain the phases and objectives of the project
  41. Show familiarity with (not actually compute) more advanced AIE concepts by
  42. Explaining in narrative form one or more potential optimization approaches to the case study investment
  43. Explaining the use of a classification chart

Practical Experience Checklist

The candidate will be assigned the following tasks to be completed with minimal direct supervision or correction as judged by the evaluator

  1. One or more parts of the business model will be assigned to the candidate
  2. The candidate will be asked to audit the model with errors (known to the evaluator but not to the candidate) included
  3. The candidate will be asked to perform Monte Carlo simulations and VIA
  4. The candidate will be asked to perform some part of an actual empirical measurement
  5. The candidate will be asked to facilitate part of a workshop
  6. The candidate will become calibrated
  7. The candidate will complete one or more parts of the final deliverable

Each of these will be evaluated as follows:

  1. Task was completed at an acceptable level without intervention by evaluator
  2. Task was completed at an acceptable level after some intervention by evaluator
  3. Task was completed only after significant supervision
  4. Task could not be completed by candidate

The requirements for successful completion of the PEC are:

  1. At least 4 of the tasks must be evaluated as an “A” and the remainder must be evaluated as no lower than “B”
  2. “C” and “D” evaluated tasks must be repeated until they can be raised to meet the requirements in #1 above
Top View