Computing and Information Systems, 1 (1994) P

Computing and Information Systems, 1 (1994) p.59-64© University of Paisley 1994

Editorial

What is the use of software engineering?

Malcolm Crowe

The purpose of this paper is to consider what purposes of software engineering should serve, how it fails at present to achieve those purposes, and to argue for some directions for the future. The discussion begins by reviewing the objectives of software engineering as set forth by a selection of authors, and continues by taking the view of the users of information systems (as reported by another set of authors) to serve as a critique of these aims. The concluding sections of the paper address the ways in which software engineering at present fails the user community, and makes some suggestions for the future.

1. Introduction

This paper will argue that software engineering as a discipline has tended to focus on the needs of software engineers, rather than on any value it maight have to society as a whole. Perhaps this originated from a naïve belief that software was so intrinsically necessary to society that anything done to help software engineers would automatically benefit society.

However, there are now several reasons for questioning this belief.

Many users prefer to minimise their reliance on the services of software engineers, and instead get their software in packaged pre-written form. To them, software engineers are to computing what tailors are to clothing: rarely useful, always more expensive, and often disappointing.

Other users view the software engineer’s obsessive concern for precise requirements and acceptance criteria as designed more to protect the interests of the software engineer at the expense of the customer. To them, software engineers are like banks and insurance companies in their use of small print.

At a more technical level, the yardstick of benefit to software engineers has resulted in ever more high-level tools. The inefficiency in the resulting products has translated for the user into a need for faster and larger computers.

The conclusion of this paper is that software engineers need to foster the development of new attitudes within their profession and among users; and to respond imaginatively to the challenge posed by the changing situation. It is still true (never more so) that software is necessary to society; software engineers need to reposition themselves in relation to meeting that need.

2. What software engineerS SAY

The purpose of this section is to consider, from the standpoint of various writers on software engineering, what its goals are, that is, what the discipline itself aims to achieve. To anticipate the conclusion somewhat, the purpose of software engineering methodologies is usually presented as making the production of software products more efficient and error-free given a complete statement of requirements. Thus the purpose of the discipline appears to be to increase the profits of software development organisations, in a variety of ways: by improving the contractual basis of software development, and by making the activities of software producers more efficient and profitable. Software engineering seems to have relatively little interest in devising useful generic products (which is more akin to invention), or in increasing the profitability or quality of life of the user organisations and individuals.

As its focus is on the goals of software engineering. this paper will only incidentally be concerned with definitions of what software engineering is. The IEEE definition (IEEE, 1983):

The systematic approach to the development, operation, maintenance, and retirement of software.

will do quite well enough here.

The original Garmisch conference in 1968 (Naur and Randell, 1969), for which the term “software engineering” was coined, began from a set of growing problems that was referred to as the software crisis. The problems identified included the non-completion of some large software projects at that time, and the cost overruns, late delivery, lack of reliability, inefficiency, and lack of user acceptance of many software projects that were completed.

It is easy to select from standard textbooks clear statements about the nature and goals of software engineering. Fairley (1985) suggests that

the primary goals of software engineering are to improve the quality of software products and to increase the productivity and job satisfaction of software engineers.

Gilb (1988) observes that

The software engineers only have a responsibility for meeting the limited design objectives laid down for them by the softect [software architect].

Simons (1987)has a section entitled “Aims of Software Engineering”:

The methodology is based on the established techniques, methods and controls that have evolved in hardware development. There are differences between hardware and software engineering (for example, the absence in software of critical ‘stuff’ out of which the product is fabricated), but also to substantial similarities that allow a common approach to such aspects as planning, development and management control. In particular, software engineering:

aims to develop a well-defined methodology that can be applied to the software-development life cycle;

seeks software components that can document the various stages in the life cycle;

seeks to define ‘milestones’ that can be reviewed in a systematic way during the task of software development.

Bell et al (1987) give

the reasons for having something called ‘software engineering’

by simply listing the goals of low cost of production, high performance, portability, low cost of maintenance, high reliability, and delivery on time.

As a result of this perspective, software engineering as a technological discipline has evolved a number of distinctive strands:

Ways of conducting or verifying the implementation of specifications (preferably “automatically”) into a product or prototype

The development of tools to express requirements, designs, and algorithms and transform them from one form to another

The development of testing and verification strategies

The establishment of an overall perspective on the “life-cycle” of software, and how this can be managed and coordinated

Cost and quality estimation

While all of these are useful, none of them is of direct benefit to the customer. Many cost metrics are based on lines of code (e.g. Boehm 1981), which encourages software engineers to adopt wasteful programming methods, and to recharge for the same design and implementation if it can be reused in another project.

3. The viewpoint of the customer

Friedman (1989) considered the attempts by software engineers to tackle the problem of user needs. He considers a wide variety of user problems under the heading of user relations, noting that during at the start of the 1980s, one of the major problems was in requirements specification. According to Jackson, the common complaint of users of data processing systems was that ‘the analyst doesn’t understand the business’. This springs from a failure to agree on an explicit model of the real world (Jackson, 1981, p.186):

... it is not surprising that the delivered system so often fails to meet the user’s needs while the user’s view of reality differs from that of the system builder.

Friedman goes on to identify the life-cycle model as a reason for poor communication He quotes McCracken (1981) as observing that specifications do not improve communication because user requirements are hard to verbalize and they are likely to change between the requirements specification phase and the operationalization phase of the system life cycle:

Some one has suggested that the plaintive cry of the user is ‘I can’t tell you what I want, but I’d recognize it if you showed me one!’ (p.447)

During the 1980s (the ‘decade of the user’), five strategies were commonly pursued for dealing with user relations problems: user involvement in systems development, end-user computing, decentralisation of information centres, prototyping systems, and changing the skill base of systems developers.

According to Friedman (1989), reviewing the literature on user participation schemes would indicate that it has not been the great panacea, though it led to certain positive results. The main complaint from users in such experiments was that involvement and influence are not the same. Users found that they have either been bewildered by jargon and cowed into quiescence, or that systems developers have tried to indoctrinate them into their way of thinking. The main complaint from systems developers was that users were unappreciative of their efforts (e.g. Mambrey and Schmidt-Belz, 1983):

The requirements of the users were sometimes regarded as being very simple and the designers shrinked from realising such ‘primitive, short-sighted and obsolete solutions’ ... After lengthy, troublesome preparations [systems designers] were confronted with disinterest, acceptance of the work as being trivial or criticism of marginal details.

Fundamentally, though, for Friedman, there is a difference in the perspectives of the two groups. Software developers are concerned with successful software. They conceive of user involvement as a way to achieve better-quality software, software that will really be used, that will satisfy user needs, software that will thereby enhance their own standing. Users are concerned with their overall working situation, with their jobs, with both the complete range of tasks they perform and with their relations within the overall organization. Relations with a particular new computer-based system are only a small component of these concerns.

Mainly though, the costs of user involvement were all too obvious to their managers, while the benefits were hard to quantify.

Friedman notes that the dream of users developing their own systems, of clients being both design interactors and end users, is an old one, and is often expressed as a strategy to remove the need for departments of specialist programmers and analysts altogether. That dream of end-user computing is being increasingly realized.

The reasons for this success story include wider computer literacy, the impatience with long waiting times for new applications leading to users going it alone, and the decentralization of computing to user departments. The growth of end-user computing appears now to be so great because the new management group of users supported by specialised systems is so much more visible and powerful than earlier groups.

4. Learning the lessons

4.1 Fitness for purpose

At one level, during development, any model of the intended use of a software product will always be only in the designer’s mind, and others involved in the design process will have a different concept of it, try as they may to coordinate their ideas. Even if implementation proceeded entirely to the designer’s satisfaction, testing, installation or use of the resulting artefacts would change the situation in the user organisation in such a way that the resulting information system (including the new artefacts) will be different from what was intended. The cyclical action research paradigm seems to be appropriate here: investigate, make a plan, carry it out, and investigate the changed situation.

Ricoeur (1970) has characterised the hermeneutic approach to investigation as one of constructive suspicion. It pays for the analyst to be suspicious of the view of the organisation, its needs and procedures, presented by the client, or of the people interviewed in the course of analysis. It pays for the client to be suspicious of the grandiose plans suggested by the engineer, to question their appropriateness to the culture of the organisation, and to consider whether a different method of implementation might be less troublesome in the long run. It pays for user and implementor during the difficult phase-in process to be suspicious of the reported faults or improvements, and of any apparently over-simplistic or over-complex analyses of how things should be done. It pays, as it has always done, for members of organisations to be suspicious that others’ objectives or agendas are incompatible with their view of what is best for the organisation. In all cases, such suspicion can motivate discussion and debate, leading to a strengthening of shared culture at the relvant levels, if sometimes at the expense of cherished ideas or individuals.

In a wider sense, the problems of interpretation posed by the differing information requirements for strategic decision-making and order-processing, or safety and machining (to take some extremes) may be much harder to solve. In one sense, of course, the organisation’s information system already deals with this problem: for is the data representing this information not already being processed in the various offices of those responsible? It is rather the challenge of finding good models for the relationships between this data, so that better support can be given in the form (for example) of extraction or updating of the relevant figures.

4.2 What is appropriate to implement?

Ultimately, for the computing professional as for management, it is this kind of issue, bearing on what can be automated: what can safely be hidden, that is at the heart of all problems of design and implementation. In an organisation of any size, managers have already delegated much of the interpretation of information to junior staff, and from their point of view, delegating it to a computer may not seem a large step. But it is foolish to ignore the intelligence of even the most junior clerk: over-simplistic automation is what has always led to stupidities such as bills for £0.00 and other traditional problems of computer implementation. Any analysis of the procedures of any organisation must inevitably simplify or select what to describe, or what scenarios are likely; just as any discussion of information presupposes a shared culture.

From this analysis, it would seem likely that new equipment or procedures will prove difficult to accept the more it is the case that

the existing work patterns are impossible in the new situations

the human interrelationships (e.g. lines of responsibility, workload) need to change

new and unfamiliar procedures need to be adopted immediately

the new equipment or procedures are incompatible with the existing documentation, so that some duplication of effort is required during the changeover

the new procedures are more inflexible than the old.

The magnitude of these difficulties should not be underestimated. For example, some quality management procedures (Tageuchi, ... ) that are popular and effective in Japan and some Western countries have proved quite unacceptable in France where the culture of management and associated notions of responsibility are different. Many writers have reported on what seems to the software engineers as “sabotage” of the new procedures, where users have found ways of working round the new restrictions rather than ”giving the new system a chance”.

4.3 Organisational change

On the other hand, radical innovation can sometimes be successfully carried out, with sufficient determination on the part of management. For example, the introduction of new printing procedures and plant by Times Newspapers in the 1980s was carried out by completely replacing the printing personnel. The expected benefits of such major changes must be weighed against the likely costs in disruption and loss of goodwill.

As a general rule, then, it seems wisest to adopt plans that can be introduced incrementally, and so command wide user support. Software engineers must resist the temptation to go for the design that is technologically the cleanest: (a) the complexity of the existing system may not be an accident, and the analyst must consider whether some complicating factors have remained undiscovered in the analysis, or not given their due weight; and (b) users’ view of aesthetic criteria (such as “cleanest”) may not match that of the analyst, so that if the change is felt by them to be unnecessary (“change for the sake of change”), the reaction of the intended users may frustrate the grand design.

Phased implementation can often help greatly. There seem to be three main advantages: first, the above types of user resistance can be tackled in stages, so that introduction of each phase is acceptable; second, experience in implementing the early phases can lead both to improvements in the design and implementation of the aspects covered by them, and to improvements in the design of the later phases (the prototyping advantage); and finally, the interaction between users and implementors will lead to greater trust and understanding on both sides, not just as people, but of the problems that both sides are addressing.

4.4 Implementation approaches

There are also conclusions to be drawn on the way that implementation of software aspects is carried out. It seems well established that the old “waterfall” model of software development has contributed to bad approaches where each phase is handled by a different team, with different objectives and a culture that focusses too much on reinterpreting requirements documents and so may not put the original needs of the organisation first. A sufficient overlap of the analysis and design team with those involved in development, introduction and maintenance of the software, is required to ensure a continuity of culture; and great care is required to ensure that the implementation is not identified too closely with individuals whose view of the organisation, its operation, and its needs, might be seen or claimed to be idiosyncratic.