Assessing the Reliability of Self-assessment based Post-Completion AuditingReports
Dr. Jari HUIKKU
Aalto University School of Business
Department of Accounting
Runeberginkatu 22-24
00100 Helsinki, Finland
Visiting AUT University, Business School
Abstract
This study investigates the reliability of self-assessment based post-completion auditing (PCA) reports. More specifically, it focusses on exploring, mapping and discussing the circumstances influencing potential for PCA report manipulation. In doing this the paper draws on the combination of literatures on agency theory, actor network theory, and concept of equifinality provided originally by open system theory. The empirical evidence of this case study is based on 24 semi-structured interviews and documentation of PCA reports in one of the major European forest company. In spite of the scepticism among scholars, and CIMA’s current definition of PCA suggesting that it has to be ‘an objective, independent assessment’, it is common to conduct PCA as a self-assessment based review. The findings suggest that to consider self-assessment based PCA suspicious and unreliable is a too hasty conclusion. The phenomenon is much more nuanced. It is argued that in assessing the reliability of self-assessment based PCA various aspects have to be taken into account. The findings lend specifically support to the contention that measurability/verifiability of PCA data enhances the perceived reliability of PCA reports. Additionally, collectivity in constructing the PCA reports appears to play a major role in ensuring the quality of PCA reports.Furthermore, with regard to practical implications, it is suggested that CIMA could consider modifying its definition of PCA by loosening or re-defining the requirements about ‘an objective and independent’ assessment.
1. INTRODUCTION
This study examines the reliability ofself-assessment based post-completion auditing (PCA) reports. The PCA of capital investmentsinvolves a formal review of a commissioned investment project, focusing on a comparison between the pre-investment estimates and the actual achievements after completion (Huikku, 2007; Chenhall and Morris, 1993). PCA can be regarded as one formal control system within a company’s total management control system package, which comprises various formal and informal controls (Otley, 1999; Malmi and Brown, 2008). It appears that most of the large companies in the Anglo-Saxon countriesconduct PCA and many companies in other countries have adopted PCA as well. The following adoption rates have been reported in different countries: 1) In UK, 74% (Harris et al., 2009), 98% (Arnold and Hatzopoulos, 2000) and 79% (Neale, 1991b); 2) In USA, 88% (Farragher et al., 1999), 76% (Gordon and Myers, 1991) and 90% (Klammer and Walker, 1984); 3) In Norway, 41% (Neale, 1994); 4) In Italy, 71% (Azzone and Maccarrone, 2001); and 5) In Finland, 67% (Huikku, 2011).
Prior research suggests that the major objective for companies to implement PCA is the enhancement of organizational learning (OL) for future capital investments (Neale, 1989, 1994; Azzone and Maccarrone, 2001; Huikku, 2009). Similarly, the major perceived benefits from PCA within companies are related to its enhancement of organizational learning(Corr, 1983; Neale, 1991a, 1994: Pierce and Tsay, 1992; Huikku, 2008). In other words, PCA information has the potential to help a company avoid previous mistakes and to systematically identify successful processes that can be repeated in the future investment projects (Shapiro, 2005; Northcott and Alkaraan, 2007).Specifically, PCA can help companies improve the accuracy of underlying assumptions and goals in their investment planning material (Chenhall and Morris, 1993; Huikku, 2008). Contributions to PCA literature suggest that PCA information can also be conducive to learning for capital investment processes in general - not merely for project-specific investment activities (Mills and Kennedy, 1993; Huikku, 2008). PCA information may, for example, trigger improvements in capital investment procedures and instructions. Learning is of utmost importance for enhancing success in capital investment which in turn greatly affects the extent to which a company can achieve its strategic objectives. In addition to enhancing OL, accountability related issues such as measuring performance of an investment project, enhancing the integrity of investment appraisals and evaluating/rewarding personnel have been reported as managerial uses of PCA (Huikku, 2008; Neale, 1991, 1994; Pierce and Tsay, 1992). Performance measurement can be considered PCA’s core and prerequisite function; it supports other uses of PCA, but it is not beneficial per se (Huikku, 2008). Moreover, PCA’s beneficial role in assisting detection of underperforming current investment projects and analyzing the appropriate actions required (correction/abandonment) is perceived within companies to be of minor, but not negligible, importance (Neale, 1989; Pierce and Tsay, 1992; Huikku, 2008).
PCA reports play a major role in communicating the results of PCA in an organization, and hence, in enabling PCA’s OL and accountability functions. In a PCA report a company evaluates the success of an investment by comparing and analyzing the ex-post outcomes of an investment project with its ex-ante targets (Neale and Holmes, 1991). In order to be useful for OL and accountability purposes, the PCA reports have to be relevant and reliable.[1] In PCA context relevance could be interpreted to mean that information is relevant when it helps managers to learn from the PCA for future capital investment (OL uses) or to support PCA’s accountability functions (accountability uses). Specifically, in investigating the quality of PCA reports, this study focusses on reliability of PCA reports by addressing its three fundamental characteristics: completeness, neutrality and freedom from errors. In assessing thereliability of PCA reports,professional competence and objectivity of the person or team conducting PCA and making PCA reports will be discussed because they have been suggested to be heavily associated with the quality of the PCA reports (e.g. Gulliver, 1987: Dillon and Caldwell, 1981; see also e.g. Eilifsen et al., 2010, p. 149 in internal auditing context). The term objective is understood to mean existing outside of the mind, having a separate or independent existence. In most contexts, objectivity means fairness and impartiality; someone who is objective has not allowed prejudice or self-interest to distort judgment (Porter, 1996, 4). Objectivity refers to external reality independent of a person who perceives it (Ijiri, 1967).The concept of independence of PCA conductor will also be addressed in assessing the objectivity, and consequently reliability of PCA reports.
Researchers have different opinions about who would be the most suitable person or team to conduct PCA. According to one approachemphasising the professional competence of PCA conductors (e.g. Dillon and Caldwell, 1981), the compilation of a PCA report requires the contribution of people with detailed knowledge. Other researchers (e.g. Gulliver, 1987) contend that objectivity can be achieved only by using outside people or a team that has not been involved in the investment project. Lumijärvi(1990) reports in his case study that the objectivity can be deteriorated if the investment group members were allowed to review their own investments. They utilised their information advantage to manipulate PCA figures and exaggerated performance estimations. In a similar vein, Cheng et al. (2009) suggest that there are risks of biased information sharing if the project personnel conduct the PCA itself. Even though there are obvious risks for reliability, for example Farragher et al. (1999) and Huikku (2011) report that in most of the companies PCA is conducted by persons or teams with prior involvement in the project. In fact, interestingly, this common practice appears to contrast sharply to CIMA’s (2005, 60) definition of PCA emphasising objectivity and independence of PCA:
“Post-completion audit is an objective, independent assessment of the successof a capital project in relation to plan. Covers the whole life of the project and provides feedback to managers to aid the implementation and control of future projects.”
Although the common practice in companies to allow staff to review their own investment projects has been recognised as a major threat for the reliability of PCA reports, there are still major gaps in the research. To the best of my knowledge, the relationship between PCA conductor and the reliability of PCA reports has not been a primary focus of any empirical study. One plausible reason for this is the difficulty to get an access to sensitive data potentially revealing embarrassing issues about dysfunctional behaviour within a company. Dysfunctional behaviour can be described as actions in which the subordinate attempts to manipulate elements of an established control system for his own purposes (e.g. Jaworski and Young, 1992).
In spite of the significant role that PCA conductor choice has for the reliability of PCA reports, our knowledge about this important relationship is in its infancy. Nevertheless, Lumijärvi (1990; case study) and Cheng et al. (2009; laboratory experiment study) suggest that self-assessment based PCA involves risks for reliability in terms of information manipulation. Consequently, motivated by Lumijärvi’s and Cheng et al.’s studies, our aim is to examine if, how, and under which conditions companies jeopardise the reliability of PCA reports when PCA is conducted by business units making the investment. In examining potential for information manipulation, we draw on agency theory related literature; specifically on Birnberg et al’s (1983) framework about information manipulation behaviour that groups it into six categories: smoothing, biasing, focusing, filtering, gaming, and illegal acts.Because of asymmetric information distribution, managers may be in a position to play games in the capital budgeting process. They may use their information advantage to enhance their self-interest objectives in order to receive the desired funds for example by focusing on certain aspects of information, filtering information or even distorting information (Lumijärvi, 1990).
In explaining reliability of PCA reports the paper draws also to the management control package literature (see e.g Malmi and Brown, 2008) and specifically on the notion of equifinality. In an organizational context, an equifinal situation means that the same final state can be achieved by multiple organizational structures, even if the contingencies faced by the organization are the same(Gresov and Drazin, 1997). In PCA context an equifinal situation occurs in performance measurement if other control mechanisms can act as alternatives to PCA (Huikku, 2007). Hence, the PCA reports based on self-assessment do not constitute the only source of information about the success of an investment project (Huikku, 2008). Additionally, the notions about collectivity and heterogeneous networks provided by actor-network literature (Latour, 1986, 2005) will be utilized in explaining the perceived reliability of PCA reports. In line with Latour’s ideas, Mouritsen (2011) suggest that collectively produced numbers in heterogeneous networks comprising both human and non-human actors are perceived more “objective”.
This study contributes to the PCA literature by extending the discussion about relationship between the reliability of PCA reports and PCA conductor.More specifically, it explores, maps and discusses aspects diminishing (increasing) potential for manipulation of self-audited PCA reports. In doing this, the paper contributes also to agency theory based discussion about gameplaying behaviour in capital budgeting. Additionally, the paper extends the notion of equifinality to explain why self-auditing does not necessarily jeopardize the reliability of PCA reports. Furthermore, we discuss and challenge the appropriateness of CIMA’s current definition of PCA suggesting that it has to be “an objective, independent assessment”. Moreover, with regard to the practical implications of the study, enhanced understanding about the aspects affecting the reliability of PCA reportsmay help organizations to design their PCA systems more appropriately.
In this study, PCA is defined as follows. PCA is a formal review of a completed investment project fulfilling the following criteria: (1) it takes place after an investment has been completed (commissioned) and has begun to generate cash flows (or savings); (2) PCA reporting is at least partly focused on a comparison between the pre-investment estimates of an investment project and the actual figures and achievements after completion; and (3) PCA is systematic and regular, and there are instructions related to it. This definition is congruent with that suggested by Gadella (1986), Pierce and Tsay (1992),and Chenhall and Morris (1993), but is more explicit with regard to criterion (3).[2]The empirical evidence of this case study is primarily based both on the 24 interviews in one of the major European forest industry corporation and the examination of its latest PCA reports related to major investment projects.
The remainder of this paper is structured as follows. Section two presents the literature that forms the basis for this study, and Section three describes the research method. The empirical results are presented in Section four and discussed in Section five. Section six concludes the paper.
2.Literature
Reliability of PCA reports
A PCA report typically include the following elements (see e.g. Huikku, 2001 and 2011; Azzone and Maccarrone, 2001; Neale and Holmes, 1991): 1) description of the investment project, 2) followup on the cost budget, scheduling, and technical specifications, to see that they are progressing according to plan, 3)comparison of ex-ante and ex-post investment calculations, 4) comments on the achievement of objectives. Additionally, companies may include the following elements in their PCA reports: proposals for action (suggestions, helpful hints, lessons learned), comparison of non-monetary targets and actual outcome, and attachments including more material. Mukherjee (1988) and Neale and Holmes (1991) have presented models for PCA reports. According to Huikku (2011), less than half of the companies studied in Finland used a standard format for PCA reports. In order to ensure comparability between ex ante and ex post calculations (e.g. NPV, IRR, Payback period), it is appropriate to use the same capital budgeting techniques in both phases. Because PCA is typically conducted already much before the end of the life-cycle of an investment (e.g. one year after commissioning), PCA calculations include uncertain forecasts about the future cash flows of investments (Neale and Holmes, 1991; Huikku, 2011).
In this study we will make an attempt to assess the reliability of PCA reports by addressing its three characteristics: (1)completeness,(2) neutrality, and (3) freedom from error. A complete PCA report is expected to include all information necessary for a user to understand the phenomenon being depicted, including all necessary descriptions and explanations. Neutrally depicted PCA report is without bias in the selection or presentation of information. It is not slanted, weighted, emphasized, de-emphasised or otherwise manipulated to increase the probability that information will be received favorably or infavorably by the users. A PCA report is free from errors if there are no errors in the description of the phenomenon (i.e. the success of an investment project).
It appears that the reliability of PCA reports is not perceived as a major challenge in the previous studies. This is interesting, because commonly adopted self-assessment based PCA does not comply with CIMA’s PCA definition or suggestions by the scholars (Lumijärvi, 1990; Cheng et al., 2009) pinpointing risks of biased PCA reports. According to Linder’s (2005) literature review of empirical studies about the problems of PCA, the lack of reliability has not explicitly been mentioned among the 23 different problem areas listed. In line with this, Huikku (2004 and 2008) found in his explicit investigation of this issue that the companies do not perceive the lack of reliability of PCA reports as an essential problem even though they commonly conducted self-auditing based PCA.
Choice of a PCA auditor
Relatively many authors have in their normative writings covered issues related to the choice of PCA auditor and its relationship to the PCA quality, whereas empirical research is scarce.Normative literature focuses specifically on weighing the pros and cons of different PCA auditor choices from professional competence and objectivity point of views. The polar alternatives for PCA auditor choice are that people earlier involved with the investment project conduct PCA or outside persons with no prior involvement conduct it.The writers suggesting thatit would be appropriate if PCA is performed by the same persons that were responsible for the justification of the project emphasize competence related issues (Dillon and Caldwell, 1981). Self-auditors have the most intimate knowledge and understanding about the objectives and outcomes of the investment project which in turn helps efficient data gathering and comparability in analyzing (Holmes et al., 1991). Additionally, PCA auditors could obtain direct feedback about the success of their investment projects and reasons affecting it, and hence by reflecting own activities they can learn for the future investment projects (Dillon and Caldwell, 1981; Gadella, 1991).Direct and faster feedback may also enhance the ability tomodify an on-going investment project that is underperforming (Dillon and Caldwell, 1981). Moreover, Linder (2007) suggests that the high degree of participation of self-auditors in PCA will reduce the perceived distrust effects of PCA, and consequently enhance employee satisfaction.
According to another approach suggested in the normative literature, objectivity of PCA reports can be achieved only by using outsiders or a team that has not been involved in the investment project (Gulliver, 1987; Northcott, 1992).Self-auditors being involved in initiating or running the project may have vested interests in performing PCA. They could present the situation subjectively or even utilize their information advantage to manipulate PCA reports becausetheir motivating force may be self-preservation and avoidance of criticism by both peers and superiors in the organization (Holmes et al., 1991). Temptation to manipulation is expected to increase if the project is underperforming or the original investment appraisal has been sketchy or even defective (Neale, 1995). Additionally, Huikku (2001) suggests that room for manipulation or over-optimism in PCA calculations is higher with regard to the estimations of future cash flows than actual figures. Independent PCA auditors are expected to be more critical, to pose more penetrating questions, and to see and pinpoint more general improvement potential in the capital budgeting process (Huikku, 2001; Neale and Holmes, 1990). Even though the PCA is conducted by an independent person or group, the PCA reporting may be biased if the investment project initiator has influence over the information presented to PCA auditors (Northcott, 1992).
Proponents of independent and objective PCA auditors come out with three options: internal auditors (Gadella, 1991; Northcott, 1992; Singhvi, 1986; Holmes et al., 1991), a designated person/team (Gulliver, 1987; Neale and Holmes, 1990; Northcott, 1992; Neale, 1995), and staff from division/group or other business units (Neale and Holmes, 1990).Internal auditors have been suggested to be appropriate for conducting PCA because of their broad knowledge of accounting and finance, and auditing skills. The challenges of using completely independent persons/units are primarily related to the lack of project-specific knowledge (Neale and Holmes, 1990). Hence, data gathering and analyzing can be time consuming and difficult, and the quality of PCA reports may suffer from this. Additionally, inter-divisional competition may dilute the quality of PCA reports conducted by staff from other business units in the corporation (Neale and Holmes, 1990).