Audit Sampling


The objective of statistical sampling in an audit context can be defined as follows:


To employ random selection procedures and statistical evaluation techniques in representative testing (whether for compliance or substantive objectives) so as to eliminate the risk of bias and so as to permit quantification of the sampling confidence achieved wherever this greater objectivity is warranted. Quantification of sampling confidence in turn helps the auditor to ensure that sufficient assurance has been obtained and that the extent of the audit work is consistent from engagement to engagement.


In today's auditing environment, the auditor seldom performs audit tests on all items in an account balance or class of transactions for the purpose of evaluating some characteristic of the population. Consequently, the evidence for an account balance or class of transaction is based upon the reasoning that the characteristics found in a representative sample of a population are reasonably accurate reflections of the characteristics to be found in the whole of that population.


Determining whether or not a test of an account balance or class of transactions should include audit sampling depends on the objective to be achieved by the procedure. If the objective of testing the recorded amount of several items included in an inventory balance is to project the results of the test to the entire inventory balance, the auditor should use audit sampling. On the other hand, if the objective were to test for misstatement in only those few items without evaluating the characteristics of the inventory as a whole, the procedure would not involve sampling.


Once a decision has been made to use audit sampling, the auditor must choose between statistical and non-statistical or judgment sampling. Judgment sampling is sampling without particular regard to the parameters of a statistical sample. Once again, this decision should be based on the objectives of the procedure and audit sample to be taken. A procedure calling for an audit sample with the objective of making a judgment on the whole population would require a statistical sample rather than a judgment sample. Another factor to consider would be the lost effectiveness of using statistical rather than judgment sampling. Statistical sampling would not be feasible when sampling a relatively small population. Ultimately, the auditor should rely on sound audit judgment in determining which method to use.


When to use Statistical Sampling

Statistical sampling methods should be used when any of the following criteria apply:


1.      1.      Cost-benefit analyses support the additional costs and time required;

2.      2.      The sample errors or exceptions must be extrapolated to quantify for the population or a defensible expression of the test results is required;

3.      3.      The objective of the financial audit is to state an opinion on the reliability of the balances reported;

4.      4.      With the availability of computer software for sampling, they would be simpler to apply; and/or

5.      5.      The risk of a sampling error must be quantified.


When to use Non-statistical Sampling

Non-statistical sampling methods may be used when any of the following criteria apply:


1.      1.      They are designed to be equally or more effective and efficient as statistical sampling, while being less costly;

2.      2.      An auditor encounters a well-designed, well-controlled system, good management, well-trained employees and a feedback mechanism that highlights errors and it would therefore be extravagant to spend a great deal of time performing extensive substantive tests;

3.      3.      An auditor encounters a system that is so weak (e.g. inadequate controls and/or procedures, insufficiently trained personnel) that no reliance can be placed on the system of internal controls and it would therefore be extravagant to spend a great deal of time performing extensive substantive tests;

4.      4.      The audit objectives are fully met by a non-statistical sample;

5.      5.      It is known that the population has no variability;

6.      6.      Examples of deficiencies are needed to support the auditorís contention that the system is weak; and/or

7.      7.      Clues are needed as to whether to proceed with a statistical sample.

In all cases, the auditor should be experienced with the sampling process and alternative sampling techniques, including statistical.


Requirements of a Statistical Sampling Plan

Statistical sampling is a particular form of representative testing in which mathematics is employed to assist judgment in the planning, selection and evaluation of the sample. Any representative statistical testing must have four chronological steps:

1.      1.      Making key planning decisions;

2.      2.      Selecting sample items;

3.      3.      Verifying sample items; and

4.      4.      Evaluating sample results.

In addition, a suitable statistical sampling plan should provide the auditor with:

1.      1.      A rational basis of making the key planning decisions required by the plan and consistent with the selection and evaluation methods to be used;

2.      2.      A valid but convenient method of selecting sample items consistent with the planning decisions already made and with the evaluation method to be used;

3.      3.      A method of projecting sample results and determining the level of audit assurance achieved which, within meaningful limits of accuracy and provided the planning and other judgmental assumptions are safe, is either rigorous or somewhat conservative.


Statistical Sampling Terminology

In statistical sampling, a sample of sampling units (such as accounts, invoices, line items, or individual dollars) is selected from a population of such sampling units. For probability theory to be applicable, the selection must be made randomly, that is, on the basis of chance. Probability theory merely measures the mathematical odds involved. Various random selection methods are possible (unrestricted, stratified, cluster, etc.), but we shall simply call any such sample a random sample. Two general sampling situations can be distinguished, attributes sampling and variables sampling.


Attributes and Variables Sampling

An attributes sampling situation is one dealing with the rate of occurrence or frequency of items in a population having a certain attribute, such as the wearing of a hat. The attribute either exists or does not. A person is either wearing a hat or is not. There is no in-between situation. Examples of attributes sampling situations are estimating the frequency of defects in ball bearings on a production line, the frequency of obsolete items in an inventory or the frequency of errors in a file of documents or in a group of receivable balances.


A variables sampling situation, on the other hand, is one dealing with variations in some measurement possessed by every member of a population. For example, everyone has a height, but individual heights vary. Indeed, an infinite number of different height measurements are possible (as opposed to the two-value, yes-or-no situation in attributes sampling). A variables sample drawn from a city can estimate the average height of its citizens based on the average height in the sample. Examples of variables sampling situations are estimating the average diameter of ball bearings on a production line, the average invoice dollar value of a group of purchase invoices or the average invoice gross profit percentage of a group of sales invoices.


In attributes sampling, the results of a random sample are expressed as a sample frequency or, in auditing, as a sample error rate (e.g., one unbilled shipment in a sample of 100 shipping orders would be a 1 percent sample error rate). The sample error rate is also the most likely error rate in the population because a random sample is likely to be representative of the population. Of course, the true but unknown population error rate may differ from this most likely error rate. The auditor can, however, if he or she has chosen the sample size properly, have a predetermined level of sampling confidence (sometimes called reliability) that the true population error rate does not exceed a determinable upper error limit. Sampling confidence, the complement of sampling risk, is merely the statistical quantification of the degree of assurance previously discussed (e.g., if sampling risk is 5 percent, then sampling confidence is 95 percent). The spread between the most likely error rate (sample error rate) and the upper error limit is called precision.


In variables sampling, the results of a random sample may be expressed as a sample average (e.g., the average invoice value in a sample of 100 invoices might be $500). The sample average is also the most likely population average. The auditor can, if the sample size has been chosen properly, have a predetermined level of sampling confidence that the true population average falls within the range of this most likely average plus or minus a specified precision.


Applicability of Attributes and Variables Sampling to Auditing

Both attributes and variables sampling techniques are in use today. There are arguments in support of a preference for either attributes and variables techniques. The applicability of these techniques depends upon their consistency with audit test objectives (both substantive and compliance).


Substantive Tests:

The objective of substantive tests is to provide an appropriate assurance of detecting the possibility of a material total of monetary error should such a material total actually exist. Monetary errors occur with a relatively low frequency in most accounting populations, but their magnitude when they occur varies. Only a few receivable balances may contain monetary errors, but the few errors that occur may vary from, say, $0.10 to $10,000. At first glance, therefore, substantive test objectives do not fit neatly into either an attribute or a variable situation. Attributes sampling is most simply used to measure frequencies, not values. Variables sampling is most simply used to measure averages of some variable possessed by every unit of the population. What the auditor needs, in contrast, is a technique of measuring the varying values of low-frequency errors. Both attributes and variables sampling methods can be adapted, however, to meet the unique requirements of the low-frequency-varying-value error situation.


Attributes sampling can be adapted to yield dollar values by one of the techniques of stratified boundary pricing, proportionate sampling or dollar-unit sampling. Variables sampling can be adapted to the audit situation either by difference and ratio methods or by mean-per-unit (direct-extension) methods.


 Compliance Tests:

The objective of compliance tests is to provide an appropriate assurance of detecting a multiple materiality value of transactions, each subject to a critical compliance deviation should such a total value actually exist. Expressed in this form, the compliance objective offers the same sampling difficulties as the substantive objective. Compliance deviations typically occur with low frequency, though not usually as low as for actual monetary errors. When compliance deviations occur, however, the value of the transaction subject to the deficiency may vary, say, from $0.10 (in which case the deficiency hardly matters) to $10,000 (in which case the deficiency may be significant). To meet this type of compliance objective, either attributes or variables sampling would have to be adapted in one of the manners listed above for substantive tests.


Types of Statistical Sampling Plans


Attributes sampling plans are:

1.      1.      Physical-unit attributes sampling (classical attributes sampling);

2.      2.      Stratified attributes boundary pricing;

3.      3.      Dollar-unit sampling; and

4.      4.      Proportionate attributes sampling (cumulative monetary amount sampling).


Variables sampling plans are:

5.   Difference, ratio and regression methods;

6.   Stratified attributes boundary pricing;

7.   Mean-per-unit method (direct-extension method); and

8.   Stratified mean-per-unit method.

(1 & 2) Neutral selection (disregarding value) followed by attributes-type evaluation.

(3 & 4) Value-oriented selection with attributes-type evaluation.

(5 & 6) Neutral selection or, if stratified, partly value-oriented selection followed by variables-type evaluation applied to errors.

(7 & 8) Neutral selection or, if stratified, partly value-oriented selection followed by variables-type evaluation applied to item values.


The technique of dollar-unit sampling (or similar plans) may be conveniently used in all audit sampling applications except those few cases where the population to be tested is not quantifiable in dollars. Dollar-unit and similar plans (such as cumulative monetary amount sampling) probably account for the largest number of individual substantive statistical sampling applications.


       Consistency with Generally Accepted Auditing Standards


Assurance that Bias is Avoided:

The risk of bias in judgmental testing can in some applications be real. There is a natural human tendency to favor (perhaps unconsciously) easily accessible selection points. If the less accessible population items happen to be the ones in error (their very inaccessibility may be related to their being in error), the sample bias could lead to seriously misleading conclusions. It is also possible that judgmental selection will avoid, say, the first and last items on any page on the grounds that such items do not seem as random as others. Of course, a true random sample will select such items some of the time. If there is some systematic reason for initial or final items on each page being more error-prone, a judgmental test avoiding them could again lead to misleading conclusions. There may also be an instinctive tendency in a judgmental test to make proportionately more selections at the beginning when the auditor is fresh than toward the end when tired.


Assurance that Sample Size is Sufficient:

Far more important, however, is the assurance which statistical sampling provides that the sample size is sufficient to warrant the conclusions expressed. Whatever the abilities of human judgment in assessing qualitative factors, such as relative strength of internal control or reasons for a given error encountered, these abilities are noticeably less in assessing quantitative factors, such as how much testing is enough or how high an error frequency might really be. Our common sense seems to be less than perfect when it comes to assessing odds (a deficiency not unrelated to the popularity of lotteries). For example, if an average group of people is asked to estimate the chance of obtaining three heads out of six tosses of a fair coin, the most common (and indeed intuitive) answer is 50 percent, though a wide range of answers within the group can be expected. In fact, the chance is only 31 percent. If groups of auditors are asked how many receivable accounts must be confirmed to provide a high degree of assurance of detecting a material error if present, similar discrepancies are likely. The most important benefit which statistical sampling offers is the reduction, through the use of mathematical aids to judgment, of the risk of over-auditing or under-auditing.


Of course, where the incremental benefit of converting testing techniques to a statistical basis is disproportionately costly, the use of statistical sampling would not be justified despite these risks. Undoubtedly there are many audit tests where this is the case and where the use of judgmental testing is thus the only responsible course to follow. But there are also many other audit tests where there is no additional cost to statistical sampling or where the cost is slight in comparison with the benefits of the greater objectivity in determining extent, selection and evaluation by statistical means. Suggested criteria for making this cost-benefit decision were discussed earlier.


It is a guide to judgment, not a substitute for it. In the end, it is the auditorís responsibility to choose those tests, test extents and testing techniques, which in their professional judgment are sufficient to satisfy generally accepted auditing standards.


Office Standards for Sampling:

Again, in the end, it is the auditorís responsibility to choose those tests, test extents and testing techniques, which in their professional judgment are sufficient to satisfy generally accepted auditing standards. The purpose of this standard is to provide guidelines for the proper use of audit sampling techniques in University and University-related audits.


Statistical and non-statistical sampling techniques are mutually exclusive tools to be used as dictated by specific audit conditions. The objective in sampling is to infer conclusions about certain characteristics of a given population, without examining the entire population. The selection process (statistical vs. non-statistical) should be the result of considerations and decisions discussed below and does not affect the audit procedures performed.


There is no difference between statistical sampling and non-statistical sampling in the execution of a sampling plan, nor does the approach affect the competence of the evidence obtained or the auditorís responses to detected errors. Selection between statistical and non-statistical sampling should be made after an evaluation of the audit objective(s) and the advantages and disadvantages of statistical and non-statistical sampling.


Reporting of Results:

In reporting observations and conclusions based on the results of statistical sampling, the levels of both precision and confidence for the sample should be reported. Standard report language for reporting the results of testing using statistical audit sampling techniques includes phrases such as:


"Based on the results of our statistical testing, it is our opinion that with a xx% confidence level (reliability), the error rate for the population does not exceed x."  In reporting observations and conclusions based on the results of non-statistical sampling, the report language must be clear that a non-statistical or judgmental sampling technique was used or the report should be silent on the sampling/testing technique as further addressed in the following section, "Guidelines for Concluding Using Non-Statistical Sampling."


Guidelines for Concluding Using Non-Statistical Sampling:

Non-statistical sampling should not be used, in the report or formal conclusions, to estimate the number or value of items in a population that were defective or improperly processed. Non-statistical or judgment sampling is subjective. If no errors are found, the auditor may be able to conclude there is no basis for examining the population further or for suspecting any material error. There is no statistical basis for concluding that the auditor has adequate assurance that the error rate is acceptable/unacceptable or even reasonable. What can be concluded, however, may be sufficient for the audit purposes.


Documentation of Audit Sampling:

Workpapers evidencing sampling (statistical and non-statistical) techniques should include, at a minimum, comments on the following items:


                 Identification of the controls or attributes being tested;

                 Sampling approach (e.g. attribute, variable, judgmental or other);

                 Description of the population from which each sample is selected (e.g., size, homogeneity, etc.);

                 For statistical samples, predetermined confidence level and precision level (The following criteria ranges for sampling compliance matters are suggested: for confidence level or reliability - 95% to 99%; and for precision or tolerable error rate - 2% to 5%. Note: Lesser confidence and precision levels may be acceptable depending on the audit objective(s).);

                 Calculation/determination of the sample size;

                 Sample selection method;

                 Record of the tests performed;

                 Analysis of errors; and