Dell Embedded Systems Management Implementations

Enterprise Systems Group (ESG)

Dell OpenManage™

Systems Management

Dell White Paper

By Timothy T. Murphy

July 2002

Contents

Executive Summary

Introduction

Introduction to ESM

Major ESM Components......

Intelligent Platform Management Interface (IPMI)

The Evolution of Dell’s Embedded Systems Management

ESM3 - High Level Architecture & Features

ESM3 - Features

ESM4 - High Level Architecture

Platform Configurations

Embedded Remote Access (ERA)

Embedded Remote Access/Option (ERA/O)

Intelligent Systems Management for Dell Servers

Figures

Figure 1: Embedded Systems Management Infrastructure

Figure 2: ESM3 Architecture for Interacting with the Platform Hardware

Tables

Table 1: ESM3 Supported Features


July 2002Page 1Dell Enterprise Systems Group

Section1

Executive Summary

Embedded Systems Management (ESM) is an evolving computer industry architecture that encompasses both hardware and software features of computer systems, and plays a crucial role in the overall management of those computer systems within an enterprise’s system management framework.

This paper provides an overview of the major ESM components and related industry-standard architectures, and describes Dell’s implementation of ESM. It explains how Dell’s ESM integrates with those industry-standard architectures and supports effective management of modern Dell enterprise class servers in heterogeneous environments.

The paper also provides a differentiation between the major feature sets of recent Dell ESM architectures, and the remote access offerings that use them.

Section 2

Introduction

Although the term ESM (for embedded systems management) may strike a familiar chord, many users may not be familiar with its benefits or capabilities. The following examples highlight a few of the advanced system management capabilities that Dell’s ESM brings to the table:

  • Lights-Out Capabilities
    In the event that a server has failed, Dell’s ESM can help support personnel to interact with the server and bring it back online, even when the base operating system (OS) and applications are inaccessible.
  • Inventory Reporting and Management
    ESM defines an interface to Field Replaceable Unit (FRU) and server platform information that is integrated with the actual server platform hardware, supplementing inventory management systems.
  • Ease of Server Upgrades
    Dell’s ESM demands server management interfaces that are independent of the base server hardware, allowing for rapid deployment of server upgrades.
  • Consistent Management Interfaces
    The system management interfaces provided by ESM are consistent across Dell’s entire server line.

Although this paper contains technical information, its focus is to illustrate the practical applications and benefits that are made possible by ESM.

Section 3

Introduction to ESM

The principal objective for Dell’s Embedded Systems Management (ESM) is to provide a base infrastructure that is the means for controlling, monitoring, and generating appropriate responses to changes in the operational state of a computer system’s board set and associated peripheral systems. Graphical User Interfaces (GUIs) are used to interact with ESM, and – as shown in Figure 1– Dell OpenManage™ IT Assistant, and Dell’s Remote Access controller (DRAC) are fully integrated with Dell’s ESM offerings.

Dell’s ESM is a building block for higher levels of systems management in that it:

  • Is scalable, from low-end servers to high-end.
  • Supports multi-chassis and modular configurations.
  • Supports feature integration without requiring hardware modifications.
  • Supports a common platform management configuration interface.
  • Isolates management software from platform hardware differences.
  • Supports cross platform Systems Management software.

The ESM functions operate autonomously, and independently of the base computer system processor(s) and software, so that they are available even when the server’s Basic Input/Output System (BIOS) or Operating System (OS) is unavailable.

The independent functions provided by Dell’s ESM represent a level of manageability that is built-in to the platform hardware.

An important goal of Dell’s ESM infrastructure is to standardize the platform interfaces, not the implementation, and thereby continue to support the platform innovation that has made Dell a leader in providing enterprise-class server solutions.

Figure 1: Embedded Systems Management Infrastructure

Section 4

Major ESM Components

In order to perform autonomous platform management functions, ESM requires its own embedded microprocessor. The processor runs embedded software –or firmware – in order to perform the ESM functions. Together, the processor and its controlling firmware are referred to as the Base Management Controller (BMC). The BMC is the heart of ESM.

In addition to the BMC, ESM encompasses a variety of controllers, sensors, connectors, storage components, and an internal data path that interconnects them. This internal data pathway is called the Intelligent Platform Management Bus (IPMB).

Some ESM devices have embedded controllers that can communicate with the BMC; these devices are called “intelligent” devices. Access to any “non-intelligent” ESM devices must be accomplished with the aid of an intelligent management controller that is associated with that device (i.e., a proxy). The BMC itself acts as the proxy management controller for non-intelligent ESM devices.

Each individual ESM management controller device is assigned a unique address on the IPMB, and communication between controllers is achieved by passing messages. Each IPMB message is encapsulated within a header that contains the sender’s and receiver’s addresses. The BMC has a well-known, fixed address on the IPMB. Controllers that manage several devices use sub-addresses to distinguish between the internal devices.

Please see Figure 2 on page 10 for a detailed block diagram depicting the overall ESM architecture, and the major device interconnections. Note that, in addition to the internal buses, there may be an additional external bus that is used for communication between sub-assemblies, or chassis, that reside within the same platform. This data path is called the Intelligent Chassis Management Bus (ICMB).

The guidelines that govern the connections and interaction between these components is fully described in a set of industry standard specifications that are collectively referred to as the Intelligent Platform Management Interface (IPMI) specifications.

Section 5

Intelligent Platform Management Interface (IPMI)

IPMI is actually just one of three interrelated industry standard specifications. Together, these three specifications provide guidance to computer manufacturers for implementing intelligent platform management on Intel-architecture servers:

  1. Intelligent Platform Management Interface (IPMI) - defines the messages and system interfaces to platform management hardware.
  2. Intelligent Platform Management Bus (IPMB) - defines an internal management bus for extending platform management within a chassis.
  3. Intelligent Chassis Management Bus (ICMB) - defines the external management bus between IPMI enabled systems or chassis.

The specifications are published by Intel, and can be downloaded from Intel’s website, at

The specifications are jointly owned, and actively promoted by Intel, Hewlett-Packard, NEC, and Dell. Many other computer equipment manufacturers, besides the promoters, adopt the specifications for their individual server platforms.

The historical motivation behind the IPMI specifications was to develop a methodology for maintaining and improving the reliability, availability, and serviceability of expensive server hardware.

This legacy continues today at Dell, with advanced features being implemented primarily in high-end servers, while low-end servers support the basic features.

Section 6

The Evolution of Dell’s Embedded Systems Management

An important aspect of Dell’s ESM architecture is its alignment with, and promotion of, industry standard interfaces such as IPMI. As a result, Dell servers integrate directly with the leading enterprise management frameworks – out of the box.

Recent advances in available server hardware, as well as emerging marketplace trends, have necessitated corresponding advances in the architecture and feature set of Dell’s ESM products.

The remainder of this paper describes the major distinguishing features of the two most recent ESM offerings from Dell:

  1. Dell’s third-generation ESM Architecture (ESM3). ESM3 is present in the following Dell servers:
  • Dell™ PowerEdge™ 1650
  • Dell PowerEdge 2600
  • Dell PowerEdge 4600
  • Dell PowerEdge 6600
  • Dell PowerEdge 6650
  1. Dell’s fourth-generation ESM Architecture (ESM4). ESM4 is present in the following Dell servers:
  • Dell PowerEdge 1650
  • Dell PowerEdge 1655MC
  • Dell PowerEdge 2650

Section 7

ESM3 - High Level Architecture & Features

Dell’s ESM3 architecture captures the feature set and architecture of IPMI 1.0, so it contains elements that are built into the system planar and related system devices (e.g., SCSI back-plane).

The hardware management support includes system fans, temperature probes, voltages, and power supplies. Through use of the built-in IPMB, ESM3 can interact with Dell’s Remote Access Controller III (DRAC III) to provide out-of-band management for the main system box, and any attached chassis (such as system storage).

Dell’s ESM3 interacts with the platform hardware using standard IPMI interfaces and data formats. A high-level architecture diagram showing this relationship is shown in Figure 2.

Figure 2 also depicts the standard software interfaces that ESM3 provides for higher levels of server management software applications in the enterprise Server Management framework.

Figure 2: ESM3 Architecture for Interacting with the Platform Hardware

ESM3 - Features

Table 1 lists the ESM3 supported features. These ESM3 features are supported across the entire PowerEdge server line currently available from Dell.

Feature Description / Feature Support
IPMI Compliance (v1.0 +)
Baseboard Management Controller (BMC)
System Event Log (SEL)
Sensor Data Records Repository (SDR)
Field Replaceable Unit (FRU) inventory
Intelligent Platform Management Bus (IPMB)
Intelligent Chassis Management Bus (ICMB) for multi-chassis systems / Yes
Yes
Yes
Yes
Yes
Yes
Manageability Features List for Server Instrumentation
Chassis intrusion probes
ECC memory error reporting (pre-failure alert)
POST code log support (port80)
System Event Log (SEL) support via BMC
Variable length Watchdog timer with actions of reset, power off, and power cycling
Asset tag
Service tag (support length prefix)
Processor ID (support the logical CPUID with table)
PCI slot enumeration and status (free, occupied, hot plug in/out)
Interrupt-driven block transfer mechanism for lower CPU utilization and event generation
Update firmware remotely and locally
Support for an IPMI compliant Emergency Card (DRAC III)
Support for Emergency Management Port
Fault Resilient Bootstrap processor support (FRB) in multiprocessor configurations
Time stamp clock
Able to detect and report change in system memory
Able to detect and report change in processor type or speed / Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Sensor support (via BMC)
Temperature
Power Rail Voltage(s)
Fan speed / Yes
Yes
Yes

Table 1: ESM3 Supported Features

Section 8

ESM4 - High Level Architecture

Dell’s fourth-generation ESM offering (ESM4) builds upon the base functionality of ESM3, offering:

  1. All embedded hardware management and BMC features available in ESM3.
  2. Further integration: DRAC III capabilities are integrated with the ESM4 circuit.
  3. With ESM4 Release-2, advanced virtualization features are included:

Software independent console redirect

Virtual CD and floppy support

Virtual serial and SAC support

Different ESM4 configurations are possible, so that one, two, or all three of these additional functions are included in a particular ESM4 release. For those configurations that include capabilities from areas one and two above, ESM4 is essentially a merge of the ESM3 and DRAC III features into a single hardware circuit.

ESM4 does not replace ESM3 for all server platforms, and is not the standard (or only) IPMI-compliant BMC currently offered by Dell. ESM4 provides remote management and virtualization features for high-end servers; for some Dell platforms, the most cost effective solution is to continue using ESM3. An optional ESM4 daughter card may be used on these systems to provide low-cost DRAC III remote management capabilities.

ESM4 therefore addresses the following emerging market issues:

  • Very high-density rack-mount servers have few PCI slots. This makes the traditional DRAC III approach for remote management (where DRAC III requires a PCI slot) a poor solution for these servers. ESM4 provides a remote management daughter card solution that does not use a customer PCI slot.
  • Headless operation of rack-mount systems in data centers is growing, and traditional KVM solutions are expensive, create cable routing problems for very high-density platforms, and are not integrated with server management reset/power control interfaces. ESM4 provides an embedded KVM solution that is integrated with the remote server management functions (e.g., reset), providing a superior solution for headless support and remote management.
  • High-density rack-mount data center servers currently contain CD-ROM and floppy drives. These are typically used only for initial installation and during servicing, but add significant cost to the platforms. ESM4 Release-2 provides a means to virtualize these drives, thereby providing superior remote support, and also allowing for the removal of the CD-ROM and floppy drives from some future platforms.
  • The DRAC III graphics console redirection support provides excellent performance, but is not software independent, and cannot be used with ESM4’s virtual CD-ROM to install a new OS. ESM4 Release-2 provides software independent console redirection that allows remote OS installs for customer that do not want to install special systems management software on the host server.

Platform Configurations

Dell’s ESM4 has been designed to provide server management support for two primary target platform configurations:

  1. Embedded Remote Access (ERA)
  2. Embedded Remote Access/Option (ERA/O)

These configurations are further described below.

Embedded Remote Access (ERA)

ERA denotes IPMI-compliant platforms with an embedded ESM4 circuit. The ESM4 circuit provides the BMC function as well as DRAC III remote management functions.

Since this configuration provides embedded DRAC III capabilities, there is no need for platform support of the legacy DRAC III PCI card, or an ESM4 daughter board.

Embedded Remote Access/Option (ERA/O)

ERA/O denotes IPMI-compliant platforms that have an embedded BMC and associated sensors, but also support an optional ESM4 daughter card that can perform most of the historical DRAC III functions.

The ESM4 daughter card operates identically to the DRAC III PCI card; it communicates with the platform’s BMC via IPMI messages, using ESM interfaces in exactly the same way that a DRAC III (in a PCI slot) communicates with the BMC.

Section 9

Intelligent Systems Management for Dell Servers

The affinity between system hardware and ESM firmware make the architecture of the ESM programming model critically important, not only to the computer equipment manufacturers, but to users as well: the ability to manage today’s heterogeneous enterprise networks would be severely hampered without standard interfaces to enable common programming models.

Dell’s commitment to industry standards related to Embedded Systems Management is evident in its status as an active promoter and implementer of the IPMI specifications.

Dell is actively working to improve the reliability, availability, and serviceability of its server products by paying close attention to emerging market needs, designing its ESM architecture to meet those needs, and helping to shape industry standard methodologies for managing these important business resources.


THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

Dell, Dell OpenManage, and PowerEdge are trademarks of Dell Computer Corporation. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others.

©Copyright 2002 Dell Computer Corporation. All rights reserved. Reproduction in any manner whatsoever without the express written permission of Dell Computer Corporation is strictly forbidden. For more information, contact Dell.

Information in this document is subject to change without notice.


July 2002Page 1Dell Enterprise Systems Group