AGD Audit Manual

APPENDIX B

Statistical and Non-Statistical Sampling and the Use of CAATs

B.1Introduction

This annex illustrates how an auditor can use CAATs software to select audit samples when statistical or non-statistical sampling is being performed.

There are two basic sampling methods – monetary unit sampling (MUS) for substantive testing, and attribute sampling. The latter is primarily used for compliance testing.

For example, Section B.5 discusses MUS for substantive tests of details. It illustrates how the auditor determines the sample size, selects the sample (either statistically or non-statistically), and then evaluates the sample results. If the auditor were performing a compliance with authority audit on a particular contract or project, the auditor could use the same approach to determine the sample size and select the sample.

The auditor could also use the same error evaluation process, only now the auditor’s errors would be amounts not in compliance with a particular authority, as opposed to monetary errors in the recorded amounts. The only real difference in the process is that the auditor would normally not need to deal with “taintings” – the percentage by each a particular sample item is in error – because all errors would normally be 100% errors. (We normally cannot have, for example, an invoice that is 50% approved, or a transaction that is 70% consistent with the nature of the appropriation to which it was charged. The invoice will normally be approved or not approved and the transaction will normally be consistent with the nature of the appropriation to which it was charged or not consistent.)

Sampling is only one source of audit assurance. It is important for the auditor to consider the assurance that is derived from all audit procedures when determining whether he/she has sufficient appropriate audit evidence.

Note that the only difference between statistical sampling and non-statistical sampling is the method of selecting sample items. Planning requirements and the evaluation process remain the same.

This Appendix contains guidelines that the auditor can use to plan a sample. These guidelines should not replace the use of professional judgement. No set of guidelines can be expected to be valid on all audits.

In this Appendix, three terms occur often enough that acronyms are used:

MLEMost likely error

MUSMonetary unit sampling

UELUpper error limit

Use of CAATs

Using a CAATs tool to perform sample size calculations and error evaluations is superior to performing a manual process for a number of reasons:

(a) It will take significantly longer to do the calculations manually than to use CAATs. Error evaluation calculations, in particular, can take a long time to do manually, and can be done in seconds using CAATS.

(b) Manual calculations are complex and prone to error. For example, auditors may fail to properly rank errors from the largest tainting to the smallest tainting when doing the error evaluation.

(c) Should the auditor use planning parameters that are not are normally used or that are theoretically questionable, CAATS will either provide a warning message or will not allow the calculation to proceed. When a calculation is being performed manually, the auditor loses this protection.

(d) Manual calculations result in sample sizes that are larger than those produced by the CAATs software, further adding to the time required to perform the audit.

(e) Statistical tables do not contain every possible confidence level. Using the tables in manual calculations therefore requires rounding up the confidence level, which will further increase the sample size and further increase the time required to perform the audit.

(f) Manual calculations result in very conservative error evaluations, which could cause DAGP to conclude that the results of the work are unacceptable when, in fact, they are acceptable.

Because of the above-noted advantages, auditors are strongly encouraged to use CAATS. However, there may be rare circumstances where manual calculations are required. For this reason, the Standard Audit Working Paper Kit contains standard forms that can be used.

B.2Basic concepts and definitions

Sampling

Sampling is the selection of a sub-set of a population. The auditor takes a sample to reach a conclusion about the population as a whole. As such, it is important that the sample be representative of the population from which it was selected.

Statistical sampling

Statistical sampling is the selection of a sub-set from a population in such a way that each sampling unit has an equal and known chance of selection.

The main advantages of statistical sampling over non-statistical sampling are:

1. Because each sampling unit has an equal and known chance of selection, there is a better chance that the sample will be representative of the population than is the case with a non-statistical sample. When expressing an opinion on financial statements, having a representative sample is very important.
2. Because there is a better chance that the sample will be representative of the population, the sample results are more objective and defensible, as are the projections of those results to the population as a whole.
3. It provides a direct estimate of the maximum possible error (referred to as the upper error limit (UEL) in some CAATS).

Non-statistical sampling

Non-statistical samples are samples selected by other means which are intended to approximate the representative character of a statistical sample. However, they lack the objectivity of a statistically selected sample.

Given the advantages of a statistical sample, a non-statistical sample should, in theory, always be larger than a statistical sample. When non-statistical sampling is used, it may be appropriate to increase the sample size by 20% to 50%.

Sampling unit

The sampling unit is the specific item of which the population is assumed to be composed for sampling purposes.

As an example, consider a population of purchases for the year. Assume that the purchases are recorded by cash disbursement, that each disbursement may relate to several supplier invoices, and that each supplier invoice may relate to several purchases. In this example, the sampling unit could be:

1. Each cash disbursement;
2. Each supplier invoice within each cash disbursement;
3. Each purchase within each supplier invoice; or
4. Each Rupee of value within each purchase.

If the auditor sets an individual cash disbursement as the sampling unit, the sample selection process would be much simpler than if the auditor set an individual purchase within a supplier invoice as the sampling unit. However, by setting each cash disbursement as the sampling unit, the auditor would have to audit all supplier invoices and all purchases within each selected cash disbursement.

Physical unit

The physical unit is the specific document (cash disbursement, individual supplier invoice or individual purchase, for example) to which the sampling unit is assumed to relate.

The physical unit is normally the same as the sampling unit. The primary exception is MUS where the sampling unit is each individual monetary unit (Rupee).

Population size

The population size is the number of sampling units (cash disbursements, supplier invoices, purchases or Rupees) in the population.

The population size will vary depending on the sampling unit being used. For example, our population of purchases for the year may be composed of 16,000 cash disbursements, 30,000 supplier invoices, 70,000 purchases, and 100,000,000 individual Rupees. Depending on which sampling unit has been selected, any of these amounts could constitute the population size.

Except for small population sizes, the size of the population does not influence the size of the sample selected. For other than very small populations, the sample size is dependent on the assumed variability (error rate) of the population, on the accuracy required from the sample (determined by consideration of materiality) and the confidence level (determined by consideration of risk). Accordingly, the auditor should not think in terms of selecting a percentage of the population. Taking a fixed percentage will tend to under-sample a small population and over-sample a large population.

The following table illustrates the relationship between sample size and population size. Note how, after a certain size, the population size does not influence the sample needed to achieved the desired level of confidence.

A simple approach when using automated tools to calculate sample size, is, if the calculated sample is as large or larger than the population, then take the whole population into the sample.

Table: Sample sizes for attributes sampling, expected error rate not over 5%, confidence level 95%

Population value

The population value is the monetary amount of the population being sampled. In the above example, it would be Rs. 100,000,000.

There may be individually significant transactions that the auditor wants to examine. These could be very large transactions or transactions with high risk. Auditors often audit 100% of these transactions, and take a sample of the remaining transactions.

To arrive at the population value for sampling purposes, the auditor needs to subtract the total value of the individually significant transactions from the total population value. For example, if the auditor decides to audit all transactions greater than Rs. 500,000 and to take a sample of the remaining transactions, the total value of the items greater than Rs. 500,000 would be removed from the population value when determining the required sample size.

Sometimes the auditor does not know the population value when determining the sample size. For example, the auditor may wish to select a sample of supplier invoices for the year, and may start auditing the transactions before the end of the year. In this case, the auditor will make an estimate of the population value at the planning stage.

Sampling risk

Sampling risk is the chance that a sample is not representative of the population from which it was selected.

If the sample is not representative the auditor could reach an incorrect conclusion about the population from which the sample was selected. The auditor could incorrectly conclude that:

1. The population is not materially misstated when, in fact, it is materially misstated; or
2. The population is materially misstated when, in fact, it is not materially misstated.

When planning an audit, auditors normally try to control the first risk and ignore the second. This is because, should an auditor conclude that a population is materially misstated, entity officials will normally conduct an investigation to determine if the auditor is correct. This follow-up work would normally lead the auditor to the correct conclusion.

Confidence level

The confidence level is the degree of assurance that the auditor has that the sample is representative of the population from which it was selected. This is the converse of the sampling risk.

If the auditor uses a 90% confidence level, this means that there is a 90% chance that the sample will be representative of the population from which it was selected, and that the audit results will be correct. Put another way, there is a 10% chance that the sample is not representative of the population, and therefore the auditor may not reach a correct conclusion from the results of the work. For example, the auditor may conclude that the population does not contain a material error when, in fact, it does.

Precision gap widening and basic precision

Planned precision is the materiality amount less the expected aggregate error for the financial statements as a whole.

To illustrate using our example:

Materiality Rs. 3,000,000

Expected aggregate error in

financial statements 816,500

Planned precision Rs. 2,183,500

When planning a statistical sample, though, there is one other factor that needs to be taken into account – precision gap widening.

The reason why we need to consider precision gap widening is because, for each additional Rs. 1 in the MLE, the UEL increases by more than Rs. 1. Simply subtracting the expected aggregate error from materiality does not deal with this effect. Therefore, planned precision needs to be reduced by a further amount. This further amount is referred to as precision gap widening.

Planned precision less precision gap widening is referred to as “basic precision”. It is equal to the error that could exist in the population even if no errors were found in the sample. It therefore represents the UEL when the MLE is nil.

Basic precision and precision gap widening are calculated automatically by some CAATS.

B.3Factors affecting sample size

Possible factors and impact

Various factors will affect the sample size, as illustrated in the following table:

B.4Sample selection

This section considers the basic sample selection rules and methods. It also illustrates how these are applied in the case of (i) MUS for substantive tests of details, (ii) MUS for compliance tests (tests of internal control), and (iii) attribute sampling for compliance tests (tests of internal control).

The difference between statistical sampling and non-statistical sampling is the method of selecting the sample items. All of the planning requirements remain the same, and the evaluation process remains the same.

There are two basic sample selection rules:

1. The sample conclusion only applies to the population from which it is selected; and
2. The sample should be representative of the population from which it is selected.

The rule in (1.) applies equally to statistical and non-statistical sampling.

The auditor has a better chance of achieving (2.) with a statistical sample than with a non-statistical sample. When using a non-statistical sample, though, the auditor should still strive to ensure the sample is as representative of the population as is possible.

There are several sample selection methods that are very good at ensuring that the sample is representative of the population from which it is selected, as follows:

1. Random;
2. Fixed interval (systematic);
3. Cell (random selection); and
4. Stratified random.

These are discussed below. The example used assumes that the auditor wishes to select 200 supplier invoices from a population of 30,000 supplier invoices.

For non-statistical sampling, the objective is to try to approximate one of these methods.

For both statistical and non-statistical sampling, there normally needs to be a complete listing of the valid transactions that adds up to the total amount reported on the financial statements.

To satisfy the validity objective, there needs to be a way in which the auditor can locate the individual items that have been selected from the listing.

To satisfy the completeness objective, there needs to be a way that the auditor can go from the individual items contained in boxes and filing cabinets to the listings that make up the total amount reported on the financial statements.

In some cases the listings used may be totals of other listings. In these cases, the auditor will first make a selection from among the totals, and will then make a second selection of individual transactions from the listing supporting each selected total.