Reading: Evaluate and recommend software

Evaluate and recommend software

Inside this document

Obtaining software

Where to start

Where to look for software

What are software prerequisites?

What are system requirements?

What are system incompatibilities?

Contacting vendors

The evaluation process

Planning

Other considerations

Implementing the Evaluation Plan

Documentation and recommendations

Analysing the data

Report preparation and presentation

Summary

Obtaining software

Where to start

Once a need for software has been identified a requirement specification should be created to specify what the software is required to do. For example a business identifies that they need some office productivity software that will do word processing and spreadsheets. The requirements specification will detail all relevant information as to how word processing and spreadsheets need to work for the business. Once you have this information you can then start looking for software.

Where to look for software

Information available on software products can be accessed from many sources. These sources have varying degrees of objectivity ranging from the authoritative to the informal. Table 1 lists common sources of information and indicates a starting point for researching a product. Each source provides information aimed at different focus groups. The focus group and the authority of the source validates the information collected.

Table 1: Some sources of information regarding software products

Sources / Description / Information / Focus
Trade magazines and journals / Independent specialist or societies, representing an authority in the product area / Publications announcing release of new technology, and case studies and reviews of products / Subscribers and members, dependent on unbiased appraisals of products
IT consortiums / A group of private and public organisations linked by common technology / Publications prepared by working groups on issues relevant to members of the consortium / Concentrated on the strategies and requirements of member organisations
Industry trade fairs / Collection of different vendors promoting individual products / Product specifications and demonstrations of software and new technology / Product-oriented
Conferences / Either vendor-specific or product area-specific / Publications and presentations announcing release of new technology, software and case studies of installation / Trade and product-oriented, depending on who runs the conference
Industry networks / A loose group of organisations linked through common technology, such as user groups and news groups / Information from experiences with the adoption of various software / User-orientated, addressing common issues and problems
Key IT organisations / Organisations with a track record in market success in software and technology / Publications are organization-centred / Concentrated on the strategies and requirements of individual organisations
Independent sources / Non-authoritative individuals or groups prolific on the Internet / Views, opinions and reviews of a wide range of products / Mixed

Other sources of information may include industry colleagues, contacts and organisations similar to your own.

To help determine if the new software is suitable we will need to research the technical specifications and functionality.

What are software prerequisites?

Prerequisites are conditions that will ensure the correct running of a software application.

Software prerequisites may include:

  • specific system requirements such as hardware or operating systems [but these are listed as system requirements]
  • the prior installation of another software package
  • services such as security and access systems, networks, Internet connections, and so on.

Here are some examples of software prerequisites:

'To browse the World Wide Web using Netscape or other Web browsers, you must have a connection to an ISP (Internet service provider).'

'To use Microsoft’s Terminal Services, you must:

  • have a machine that is running Windows 2003 Servers or Windows 2000 Server,
  • have a computer with a minimum CPU speed of 550MHz, minimum memory 256MB, and 4GB free disk space'

What are system requirements?

  • To ensure the correct operation of their software, most software manufacturers will specify preconditions to the functioning of their software by recommending a minimum system configuration. The system requirements may include the hardware platform
  • the operating system
  • resource requirements (CPU, memory)
  • storage capacity for the software and data
  • hardware devices such as a mouse, CD ROM drive, printer, backup device, modem.
  • minimum specifications for hardware such as screen resolution

These system requirements ensure that software is installed and run in an appropriate environment. Many software applications can run in many different environments, but usually software manufacturers will only guarantee their software in a limited range of environments.

What are system incompatibilities?

System incompatibilities are mismatches between the software and the system configuration and or other installed software. System incompatibilities may prevent the software being installed or prevent the software from operating as expected.

For example, many applications are only available for a single operating system. Some of the powerful graphic design tools can only be run on the Macintosh platform, whilst many Windows based applications that you are familiar with do not have versions that can be run on Macintosh or Linux systems.

Another example is programs that have been written for a specific computer chip. For example Windows programs are designed and built to run on Intel processors. In order to run on alternative processors like the Power PC or Alpha processors, the software needs to be ported and compiled for that machine.

During your investigation of minimum system requirements and prerequisites, it is important to recognise any incompatibilities with your target systems.

Contacting vendors

The information collected on vendors and products can be extensive. To reduce the information to something manageable, it is assessed against the business requirements specification for software. This process identifies those products that are suitable to the business. The aim is to discard any products that are irrelevant and leave a smaller selection of products for further investigation. This usually results in two or three vendors and products that best meet the software requirements.

Information required from vendors will include:

  • a list of prerequisites for installation and operation of the product
  • a list of system requirements
  • support and maintenance services, requirements or arrangements
  • Vendor details including history and client list for references
  • Licensing details and options
  • Product pricing and costs (including ongoing)
  • Any conditions regarding evaluation of the software (installation restrictions etc)

It is also good practice to send your requirements specifications to the vendor and have them respond with how their software product addresses each requirement. Once you have all this and a copy of the software you are ready to commence the evaluation process.

The evaluation process

This is the nuts and bolts of how you will determine if the specific software meets the needs of the business. We need to plan how this will be done making sure that there is no impact on already existing business systems and processes.

Planning

Once we have sourced the software along with system requirements, prerequisites and other related information we need to plan how to evaluate the software. The plan should address:

  • The objectives
  • The stakeholders.
  • Evaluation method.
  • Resources and Timelines

Objective

What does the business want to accomplish by this evaluation? The objective should state how well-selected software will meet the needs of the business.

Stakeholders

These are the people who have an interest in the evaluation and need to know about the evaluation results. This may be senior management, specific department managers and potential users of the software. Users are significant stakeholders because they are the ones who will be left with the day-to-day operation of the software.

Evaluation method

The evaluation method outlines in detail how the software will be tested. This includes:

  • Where and how the software will be installed (hardware, location, etc)
  • Specific tests referring to the business requirement specification documents. These may be individual tasks requiring some sort of outcome to be measured, compared or evaluated. For example create word processing document, save in various locations, retrieve document may be required evaluation tasks because this criteria was stated in the business requirement statements. If the software can do this and the ease with which it can be done is evaluated by the test.
  • Who will do the testing? Generally functional testing and evaluation of software is performed by the users who require the software. These people understand the business process that needs to be fulfilled and are best suited to make judgements on the functional suitability of the software. Technical testing like installation, deployment, data backup and restores should be performed by technical staff who will maintain the software.
  • How the test results will be documented? This includes some sort of criteria and objective scale that results can be measured against.
  • Responsibilities of all involved
  • Specific evaluation of existing environment for compatibility and other issues. For example the new software may produce files that are incompatible with existing database applications.

Resources and timelines

Once you've identified all evaluation tasks, you can allocate resourcesto them, and schedulethem. In simple evaluations, this involves listing all tasks. Each task has a start date and time, duration, and resources. Resources that can be allocated to evaluation tasks include:

  • appropriately-skilled personnel
  • hardware and software
  • access to existing running system
  • system documentation, vendor support
  • system users.

Scheduling evaluation tasks requires sequencing tasks to take place when the required resources are available.

Other considerations

In developing an evaluation plan there are other issues to consider. In some cases if the original business requirement specification for the software were developed by non-technical staff. The technical requirements may not have been included. For example a Finance department may develop functional requirement specifications that address what the software should do with respect to managing the organisations financial matters, however, they cannot include technical requirements like what operating system or hardware best suits the organisation because it’s outside their area of expertise. This would be the job of the IT department. These technical considerations need to be included in evaluation planning and evaluation tasks. Some issues that influence the viability of software are as follows.

Scalability

Scalability is determined by calculating the conditions under which the software will be operating, at a point in time in the future. Conditions can cover storage requirements, transaction volumes and communication speeds. These requirements are formulated by determining the current requirements and then applying future loads and volumes. Calculations are expressed in terms of numbers of sites to service, size and the numbers of database records, the number of users and communication speeds. If software cannot scale up there may be significant operational and costs issues in the future.

Interoperability

Examining interoperability reassures the organisation that the software will work with the current installation and any other relevant emerging technology the organisation may adopt. The greatest concern to the organisation is that the adoption of a particular software solution will require the acquisition of additional compatible hardware and infrastructure. Additionally, will the adoption of the product force into redundancy existing installed software? The only way to be sure of this is by testing the software in the existing production environment. Compatibility issues also include the training of existing staff.

Compatibility

Software incompatibilities are mismatches between two or more software applications that prevent those applications from either running at the same time or from being installed on the same machine. Software incompatibilities can be caused by applications:

  • using the same shared libraries, but different versions
  • requiring the same resources
  • requiring the same pre-requisite application, but different versions.

If a common software incompatibility exists, the software manufacturer may have to either release a patch or fix which addresses the problem, or provide a “work around” which will enable the software systems to work together. Either may not produce the desired results and may impact business significantly.

Reusability

Reusability of any component in an installation adds value to that component. If a product can be used elsewhere it will prove cost effective, returning more than the original investment. Can the software service other departments or needs in the organisation?

Standards

If a product is scalable, interoperable and reusable it still may present a risk if there is no-one who knows how to use it. Software may operate within the predicted future environment, but could become non-standard. Organisations that adopt products that deviate from standards run the risk of not being able to recruit trained staff from outside the organisation.

Budget

Be aware of what your budget constraints. It would be pointless to undertake a full evaluation just to find out that you cannot afford the product. Time and effort could be spent on more viable evaluations.

Licensing

Software licensing specifies the conditions for using the software. These conditions are set by the software manufacturer or vendor and address such things as where the software can be installed, how long it can be installed for, copying and distributing the software, modification and ownership of the source code, and so on.

Consideration must be given to the type of license required and the conditions imposed. For example a single user licence may mean that the software can be either installed on one computer only or can be installed on many computers but only used by one user at any given time. Other types of licences may be concurrent user licenses where only a certain number of users can use the software at one time. Site licence or organisational licences may permit software installation and use on any number of computers at a specific site or in the organisation.

The cost of software licences usually depends upon the type of software (Commercial, shareware, freeware) and the type of licence required (single user, multiple user, site etc). The conditions of a licence may specify the duration of the licence. For example a licence may only permit software use for one year only with continuing use requiring the purchase of a licence renewal. Other conditions may specify if software upgrades and patches are provided during the licence period at no cost or at additional cost (known as software upgrade assurance).

Security

Consider how new software will fit into the organisational security requirements and policies. Will the software open or compromise access to secure data? For example, an on line purchasing application may need access to the financial information stored in a corporate database. The application its self may need to run on a web server. The mechanism used by the application to communicate with the database may leave financial data vulnerable to being viewed in transit between the database and web server. In this case Data confidentiality may be compromised.

Implementing the Evaluation Plan

With a plan developed it is now relatively simple to undertake the software evaluation. Everyone involved knows what is required, what will happen, and when.

The most important thing with any evaluation is that the evaluation process should not impact on existing business functions, processes or systems. In the case of software evaluation, the software should be installed on an isolated computer and initially tested in isolation. Following this testing the plan may call for the connection of the isolated computer to the network to test using the software via the existing network. This type of activity always contains some element of risk and the evaluation plan should address this. For example all existing systems are backed up prior to connecting the test computer to the network. The connection and testing are done after hours. There should always be some role back plan should unexpected things happen when connected to the existing live network.

Documentation and recommendations

The evaluation process requires careful documentation. The results from each evaluation task or test should be recorded so that we can compile results in to a meaningful document. The documentation allows for recommendations that can be supported and validated by test results. This documentation can then be presented to the appropriate decision makers for their consideration.

Analysing the data

The simplest way to interpret the results of evaluation testing is to create an evaluation matrix. The matrix contains all the business requirement specifications for the required software with the associated evaluation tasks. This is the evaluation test criteria.