Criteria and an Objective Approach to Selecting Commercial Real-Time Operating Systems

CRITERIA AND AN OBJECTIVE APPROACH TO SELECTING COMMERCIAL REAL-TIME OPERATING SYSTEMS BASED ON PUBLISHED INFORMATION

Phillip A. Laplante[1]

PennStateUniversity

Abstract

Matching a commercial real-time operating system to a particular application is a problem for which there is no obvious solution strategy. While factors such as estimated production volume, likely processor involved, mission criticality, cost, reliability, software support, ease-of-use, and maintainability need to be considered, oftentimes the selection is merely based on the preferences of the design team, management, or intensity of vendor marketing.

In this paper the selection of real-time operating systems is examined. A set of criteria and an associated metric are presented that rely only on published marketing information. Then, to demonstrate the application of this metric, information from several existing commercial real-time operating systems is collected and analyzed. Five hypothetical applications are then matched with the most appropriate real-time operating system, using the criteria and metric.

The contribution of this paper is a set of criteria, a metric, and a methodology for matching real-time operating systems to applications based on marketing information, without the need for intensive performance testing.

Keywords: metrics, real-time operating systems, selection criteria

1. Introduction

There is surprisingly little work on metrics for evaluating real-time systems. A literature search turned up nothing for evaluating commercial real-time operating systems (RTOS[2]) in journals. A number of reports are available either from workshops or trade magazines but these were difficult to find.

The dearth of published work on evaluation of RTOS is surprising because there is both an engineering and business advantage in some formalized methodology for evaluating candidate systems. In any case, some metrics that are documented for RTOS only involve scheduling features. For example, Buttazzo gives average response time, total completion time, weighted sum of completion times, maximum lateness and maximum number of late tasks [1]. Other metrics are based on extensive experimental procedures, [2], [3], [4], [5], [6]. Unfortunately, these are fairly limited in scope and the results of a given test cannot be necessarily imputed beyond the experimental platform. Hence, they provide only partial guidance in the selection of a commercial RTOS solution.

Unfortunately, unless a comprehensive experience base exists using several alternative commercial real-time operating systems in multiple, identical application domains, there are only two ways to objectively determine the fitness of a product for a given application. The first is to rely on third party reports of success or failure. These abound and are published widely on the Web (e.g. [3]). The second is to compare alternatives based on manufacturer’s published information from brochures, technical reports, and Web sites. Therefore, there should be some way to incorporate manufacturer’s published data into the decision-making process, especially when independent performance tests are unavailable and impractical to conduct.

The contribution of this paper, then, is an objective and flexible technique for comparing commercial real-time operating systems based on marketing information. This technique can be used, however, in conjunction with supplemental information from actual experience and third party reports.

1.1Build Versus Buy

A common question that is asked at the time of systems requirements specification is “should a commercial real-time solution be used or should one be built from scratch?” While the answer depends on the situation, commercial kernels[3] are frequently chosen because they generally provide robust services, are easy to use, and may be portable.

While commercial RTOS provide flexibility in the number of tasks supported and scheduling discipline, there are drawbacks in their use. For example, they are usually slower than the hard-wired interrupt driven model because tremendous overhead is incurred in implementing the task-control block model, which is the typical architecture for Commercial RTOS [7], [8]. Furthermore, commercial solutions tend to suffer from featureitis – too many unneeded features are incorporated in order for the product to have the widest appeal. The run-time and storage costs of these features may be excessive. Finally, manufacturers may be tempted to make misleading claims, or give best case performance figures. The worst-case response times, which are the most informative, can generally not be known. If they are known, they are typically not published because they would place the product in an unfavorable light.

For embedded systems, when the per-unit charge for commercial products is too high, when desired features are unavailable, or when the overhead is too high, the only alternative is to write the real-time kernel. But this is not a trivial task [7], [8]. Therefore, commercial RTOS should be considered wherever possible.

1.2 Commercial Real-Time Operating Systems

There are many commercial solutions available for real-time systems, but deciding which one is most suitable for a given application is difficult. Many features of embedded real-time operating systems must be considered including cost, reliability, and speed. But there are many other characteristics that may be as important or more important, depending on the application. For example, the real-time operating system largely resides in some form of ROM and usually controls hardware that will not tolerate many faults; therefore, the RTOS should be fault-tolerant [9]. Also, the hardware needs to be able to react to different events in the system very quickly; therefore, the OS should be able to handle multiple processes in an efficient manner [10]. Finally, because the hardware on which the operating system has limited memory, the operating system must also require a reasonable amount of memory in which to run [7].

In fact, there are so many functional and non-functional attributes of any commercial RTOS, that evaluation and comparison must be a subjective endeavor. Some structure, however, should be used in the heuristic decision-making. Using a standard set of criteria provides such structure.

2. Criteria for selecting real-time systems

What makes a good RTOS? In short, it depends on the standard definition of a real-time system, namely, a system in which requirement satisfaction (logical correctness) is based on the correctness of the system’s behavior in terms of data and time [7]. In essence, the RTOS must have correct and bounded behavior under all system load scenarios.

From a business perspective, there is concern about compatibility, especially in an environment where there are heterogeneous and legacy systems. Furthermore, it is critical that the RTOS support newer or standard network technologies for flexibility of product development and maintenance.

Mainstream real-time researchers and engineering practitioners seem to agree on the following desirable characteristics for real-time systems, these are:

• timeliness
• design for survival under peak load (schedulability)
• predictability
• fault-tolerance
• maintainability [1]

Therefore the selection criteria should reflect these desiderata.

Consider thirteen selection criteria, , each having a range where unity represents the highest possible satisfaction of the criterion and zero represents complete non-satisfaction. Recognizing that the importance of individual criterion will differ depending on the application, a weighting factor, , will be used for each criterion , where unity is assigned if the criterion has highest importance, and zero if the criterion is unimportant for a particular application. Then a fitness metric, , is formed as

.(1)

Clearly, a higher value of means that the RTOS is well suited to the application, while a lower value means that the RTOS is not well suited for the application.

While selection of the values for and will be subjective for any given RTOS and any given application, the availability of this heuristic metric, provides a handle for objective comparison, historical perspective and other uses. The following sections describe the criteria.

2.1 Minimum Interrupt Latency

The minimum interrupt latency, , measures the time between the occurrence of hardware interrupt and when the interrupt’s service routine begins executing. A low value of represents relatively high interrupt latency, while a high value of represents low latency.

This criterion is important because if the minimum latency is greater than that required by the embedded system, a different operating system must be selected.

2.2 Total Number of Tasks Supported

This criterion, , defines the most concurrent processes the operating system can simultaneously support. Even though the operating system may support a large number of tasks, this metric may be further limited by available memory. This criterion is important for systems that need numerous simultaneous processes. A relatively high number of tasks supported would result in, while few task supported would suggest a lower value for .

2.3 Memory Requirements

Criterion specifies the system memory required to support the OS. It does not include the amount of additional memory required to run the system’s application software. suggests a minimal memory requirement, while would represent a larger memory requirement.

2.4 Scheduling Mechanism

The scheduling mechanism criterion, , enumerates whether preemptive, round-robin, or some other task scheduling mechanism is used by the operating system. Round-robin scheduling is used to allow tasks at the same priority level to share the resources of the system. Preemptive scheduling allows high priority tasks to begin immediately by preempting the current task running. Most PC operating systems use preemptive task scheduling because an indeterminate amount of processes will be running, each with their own unique CPU requirements. However, when the number of processes is fixed, as in the case of many real-time systems, preemptive task scheduling is not always a more efficient algorithm than round-robin scheduling [10].

Kernel preemption is another feature that reduces response times and increases system schedulability. If the kernel is not preemptable, then a low priority task that is in the middle of a kernel service call would inherit the highest priority to the low priority task, creating a priority inversion. Kernel preemptability alleviates this situation.

If many scheduling mechanisms are supported and if the kernel is preemptable, then a high value would be assigned to .

2.5 Intertask Synchronization Mechanism

Criterion refers to the available methods the operating system has to allow processes to communicate with each other. Among possible choices are mutual exclusion (mutexes), binary and counting semaphores, POSIX pipes, message queues, shared memory, FIFO buffers, control sockets, and signals and scheduling. Each mechanism has advantages and disadvantages [11], [12]. Let if the RTOS provides all desired scheduling mechanisms. A lower value for implies that less synchronization mechanisms are available.

2.6 Context Switch time

Criterion refers to the time it takes for the kernel to save the context when it needs to switch from running one task to another. The context may include one or more of the following: PC pointer, stack pointer, heap pointer, I/O pointer, floating point registers, temporary storage registers, and any MMU registers. This information is needed so that the OS can return to the point of interruption without loss of information [13]. A relatively short context switch time would result in a higher value for .

2.7 Programming Environment

Application availability, , refers to the amount of software available (either that ships with the operating system or is available elsewhere) to develop applications to run on the operating system. For example, RTLinux is supported by the GNU suite of software, which includes the gcc C compiler and many freely available software debuggers, and other supporting software. This is an important consideration, especially when using an unfamiliar operating system. This criterion also takes into consideration the number of other operating systems that are compatible with the given RTOS.

Let if a large amount of software were available and compatible, while 0 would mean that little or none was available or compatible.

2.8 Portability

Criterion refers to the different processors that the OS supports. This is important in terms of portability and compatibility with off-the-shelf hardware and software. This criterion also encompasses the range of peripherals that the OS can support, such as video, audio, SCSI, and such. A high value for the criterion represents a highly portable and compatible RTOS.

2.9 Source Code Availability

Criterion refers to whether the code of the operating system will be available to the developer, for tweaking or changes. The source also gives insight to the RTOS architecture, which is quite useful for debugging purposes and systems integration. Settingwould suggest open source code or free source code, while a lower value might be assigned in proportion to the purchase price of the source code. Let if the source code were unavailable.

2.10 Software Support (warranty)

Criterion refers to the after-sale support a company puts behind its product. Most vendors offer some sort of free technical support for a short period of time after the sale, with the option of purchasing additional support if required. Some even offer on-site consultation. A high value might be assigned to a strong support program, while if no support is provided.

2.11 Cost

This criterion is directly related to the cost of the RTOS including the development license cost as well as the per target license costs. This consideration is critical because for some systems, the RTOS cost may be disproportionately high. In any case, a relatively high cost would be assigned a very low value, while a low cost would merit a higher value for .

2.12 Royalty Fee

Frequently, a per target license or royalty fee is charged for each delivered system. In the case where few targets are intended, this can be inconsequential. However, when a large number of targets are intended then the per target licensing fee can be considerable. This criterion, , is high if a relatively low per unit royalty fee is charged.

2.13 Networking Support

This criterion, , is based on a listing of what networks and network protocols are supported by the given RTOS. A high value for the criterion represents a relatively large number of networks supported.

3. Evaluation of Commercial Real-Time Operating Systems

A number of commercial RTOS are now examined based on the criteria introduced. Although the data is real, the manufacturer names are omitted as the intention is not to imply a recommendation of any product. These systems were selected based on both the availability of published information on the Web, and the knowledge that these systems have fairly widespread use. Of course, the scope of the study could have been expanded to consider many other viable commercial products. But the intent of this work is to provide a methodology and some examples of its use – not to completely assess the RTOS marketplace.

Since this work was done, clearly some of the systems performance characteristics may have changed. One must realize that any analysis is only as accurate as the data provided by the manufacturers at the time of the study[4]. This is typical of a real-world situation where there is little opportunity to test the accuracy of a manufacturer’s claim prior to making a buy decision. However, this constraint does not diminish the value of this work, which is to provide a methodology for assessment of goodness of fit for commercial real-time solutions and a particular application.

In the sections that follow, the following assumptions are made:

• For all the sample RTOS, assume that the calculations for the number of interrupt, the minimum time that it takes, and other system analysis based on the metrics chosen are performed under the same conditions, i.e. sampling, time constraints, number of processors, and etc.

• Maximum or minimum of tasks refers to the OS object, such as the MMU, device drivers, and other system tasks.

• Assume that interrupt refers to “hardware interrupt”. “Software interrupts”, together with hardware interrupts and other vectoring mechanisms provided by the processor, are referred to as “ exception handling”.

• Thread switching time is equivalent to the measurement of context switching time.

In the cases where a criterion value can be assigned, this is done. Where the criteria are “processor dependent” or indeterminate absent a real-application, assignment of a rating is postponed, and a value of * is given. This “uncertain” value is fixed at the time of application analysis. Note too that the values between tables need to be consistent. So, for example, if a 6 microsecond latency yields for RTOS X, the same 6 microsecond latency should yield for RTOS Y.

Consider commercial RTOS A. Table 1 summarizes the criteria and ratings. The rationale for the ratings are described in the following paragraphs.

.

Table 1: Summary data for real-time operating system A.