Syslog Interface to the PI System

Syslog
Interface to the PI System

Version 1.0.1.0
Rev B

1

UniInt End-User Interface to the PI System

How to Contact Us

OSIsoft, Inc.
777 Davis St., Suite 250
San Leandro, CA94577USA
Telephone
(01) 510-297-5800 (main phone)
(01) 510-357-8136 (fax)
(01) 510-297-5828 (support phone)

Houston, TX
Johnson City, TN
Mayfield Heights, OH
Phoenix, AZ
Savannah, GA
Seattle, WA
Yardley, PA / Worldwide Offices
OSIsoft Australia
Perth, Australia
Auckland, New Zealand
OSI Software GmbH
Altenstadt,Germany
OSI Software Asia Pte Ltd.
Singapore
OSIsoft Canada ULC
Montreal, Canada
OSIsoft, Inc. Representative Office
Shanghai, People’s Republic of China
OSIsoft Japan KK
Tokyo, Japan
OSIsoft Mexico S. De R.L. De C.V.
Mexico City, Mexico
Sales Outlets and Distributors

• Brazil
• Middle East/North Africa
• Republic of South Africa
• Russia/Central Asia

/

• South America/Caribbean
• Southeast Asia
• South Korea
• Taiwan

OSIsoft, Inc. is the owner of the following trademarks and registered trademarks: PI System, PI ProcessBook, Sequencia, Sigmafine, gRecipe, sRecipe, and RLINK. All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Any trademark that appears in this book that is not owned by OSIsoft, Inc. is the property of its owner and use herein in no way indicates an endorsement, recommendation, or warranty of such party’s products or any affiliation with such party of any kind.
RESTRICTED RIGHTS LEGEND
Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013
Unpublished – rights reserved under the copyright laws of the United States.
© 2003-2007OSIsoft, Inc.PI_Syslog.doc

1

Syslog Interface to the PI System1

Table of Contents

Introduction

Reference Manuals

Supported Features

Diagram of Hardware Connection

Principles of Operation

Performance

Syslog Format and Contents

Syslog Interface Message Types

PIX

Cisco IOS

Syslog General

Message Formatting

Installation Checklist

Interface Installation

Naming Conventions and Requirements

Interface Directories

The PIHOME Directory Tree

Interface Installation Directory

Interface Installation Procedure

Installing the Interface as a Windows Service

Installing the Interface Service with PI ICU

Installing the Interface Service Manually

Digital States

PointSource

PI Point Configuration

Point Attributes

Tag

PointSource

PointType

Location1

Location2

Location3

Location4

Location5

InstrumentTag

ExDesc

Scan

Shutdown

Performance Point Configuration

I/O Rate Tag Configuration

Monitoring I/O Rates on the Interface Node

Configuring I/O Rate Tags with PI ICU (Windows)

Configuring I/O Rate Tags Manually

Configuring PI Point on the PI Server

Configuration on the Interface Node

Startup Command File

Configuring the Interface with PI ICU

syslog Interface Tab

Command-line Parameters

Sample PISyslog.bat File

Interface Node Clock

Security

Windows

Starting / Stopping the Interface on Windows

Starting Interface as a Service

Stopping Interface Running as a Service

Buffering

Configuring Buffering with PI ICU (Windows)

Configuring Buffering Manually

Example piclient.ini File

Appendix A: Error and Informational Messages

Message Logs

Messages

Interface Startup Errors

Point Loading Errors

Point Debugging Messages

Run-time Error

Interface-level Debugging

Syslog Error Message

System Errors and PI Errors

APPENDIX B: PI PIX Firewall Interface Compatibility

Migration

Manual Migration

Migration Using the PI ICU

Compatibility

Count, Rate and User Points

Appendix C: Extract from RFC3164 – 4.1.1 PRI

Revision History

1

Syslog Interface to the PI System1

Introduction

The syslog protocol is a standard for logging system events over a network. It provides a transport to allow a machine to send event notification messages across IP networks to event message collectors (also known as syslog servers). OSIsoft’sPI Syslog Interface works as a syslog server for one or moredevices. The interface listens on the syslog port (UDP port 514) and collects the syslog messages sent by the devices. The interface then matches each message with the appropriate PI Pointand sendsthe required part or parts of the messages to thisPoint.

A standard format for the syslog messages is recommended by the syslog protocol. However, there are no set requirements on the contents of the syslog packet as it is originally sent from a device. Therefore, the PI Syslog Interface considers any packet received from the syslog port a validsyslog message and records the information to the corresponding PI points. In addition, the interface supports the specific syslog message formats of devices such as Cisco PIX Firewall and other Cisco devices. PI Syslog can recognize the device-specific syslog messages, parse the received packet accordingly and store appropriate information to the corresponding PI points.

The PI Syslog Interface runs on Windows NT version 4.0 SP6, Windows 2000, Windows XP, and Windows 2003 Server computers. Unless otherwise noted, the remainder of this document uses the term “Windows” to refer to all these.

PI Syslog interface requires:

• PI Server
• PI API
• Internet Explorer 4.0 or greater (The interface uses the Internet Explorer Regular Expression Engine to parse the syslog messages)

No special hardware is required by this interface.

The direction of data flow is uni-directional; that is, from the device(s) sending out the syslog messages to the PI Server.

Reference Manuals

OSIsoft

• PI Server Manuals
• PI API Installation Manual
• Regular Expressions Tutorial
• UniInt Interface User Manual

Cisco Systems

• Cisco Systems, Inc Cisco PIX Firewall System Log Messages

• Cisco – Setting Up PIX Syslog
• Cisco-System Error Messages Overview

Other

• The BSD Syslog Protocol

Supported Features

Feature / Support
Part Number / PI-IN-OS-SYSLOG-NT
* Platforms / Windows NT 4.0SP6 / 2000 / XP / 2003
APS Connector / No
Point Builder Utility / No
ICU Control / Yes
PI Point Types / float16/float32/float64/int16/int32/digital/string
Sub-second Timestamps / Yes
Sub-second Scan Classes / No
Automatically Incorporates PIPoint Attribute Changes / Yes
Exception Reporting / Yes
Outputs from PI / No
Inputs to PI: Scan-Based / Unsolicited / Event Tags / Unsolicited
Supports Questionable Bit / No
Supports Multi-character PointSource / Yes
Maximum Point Count / Point count of PI Server
Uses PI SDK / No
PINet String Support / Not applicable
* Source of Timestamps / PI Server
History Recovery / No
* UniInt-based
Disconnected Startup
SetDeviceStatus / Yes
Yes
Yes
Failover / No
Vendor Software Required on PI Interface / PINet Node / No
Vendor Software Required on Foreign Device / No
Vendor Hardware Required / No
* Additional PI Software Included with Interface / Yes
Device Point Types / Not applicable
Serial-Based Interface / No

* See paragraphs below for further explanation.

Platforms

The Interface is designed to run on the above mentioned Microsoft Windows operating systems and greater. Windows NT 4.0 requires Service Pack 6.

Please contact OSIsoft Technical Support for more information.

Uses PI SDK

The PI SDK and the PI API are bundled together and must be installed on each PI Interface node. This Interface does not make PI SDK calls.

Source of Timestamps

The clock on the computer running the PI Server provides the source of the timestamps for the data sent by PI Syslog. The interface writes a timestamp that reflects the time at which it processed the Syslog packet.

UniInt-based

UniInt stands for Universal Interface. UniInt is not a separate product or file; it is an OSIsoft-developed template used by our developers and is integrated into many interfaces, such as the PI Sysloginterface. The purpose of UniInt is to keep a consistent feature set and behavior across as many of our interfaces as possible. It also allows for the very rapid development of new interfaces. In any UniInt-based interface, the interface uses some of the UniIntsupplied configuration parameters and some interface-specific parameters. UniInt is constantly being upgraded with new options and features.

The UniInt Interface User Manual is a supplement to this manual.

Note: The interface does not use UniInt functions to write data to the PI server. For this reason UniInt parameters related to writing data (for example /q and /sn) have no effect on the interface. The interface uses the PI API function pisn_sendexceptionqx to write data to PI.

Set Device Status

For a Health Tag whose Extended Descriptor attribute contains [UI_DEVSTAT], the Interface writes the following values:

a)"1 | Starting" - the interface has started.

b)"Good" - the interface has received syslog records and has processed them.

c)"4 | Intf Shutdown" - the interface has shut down.

d)"5 | internal queue is full" - the interface's internal queue is full, and the interface cannot receive additional syslog records.

Please refer to the UniInt Interface User Manual.doc file for more information on how to configure Health Tags.

Additional PI Software

A utility for testing regular expressions (RegExpTester.exe) is included in the install of this interface. This utility is useful for testing the syntax of regular expressions before using them in a PI Point.

Diagram of Hardware Connection

1

Syslog Interface to the PI System1

Principles of Operation

The PI SyslogInterface functions as a syslog server: It listens to eitherUDP port 514(the syslog port) or any other specified port and collects the syslog messages sent by one or more devices. The Interface continuously reads the syslog port in a dedicated process thread,upon receiving each syslog packet;the interface checks the length of the received message. Because the length of a syslog packet should not exceed 1024 bytes, if a packet longer than 1024 bytes is received, the interface will truncate it to fit this limit before processing the message (see /stsp). The interface adds each message to an internal queue to be processed.The interface checks each PI Point loaded by the interface, with each syslog message. Where the syslog message matches the filter expression of this point the messages sent to PI in the format dictated by the Points Location3.

Performance

If the syslog port is receiving messages at a high rate,the interface may not be able to process the messages quickly in which case it is possible to overflow the interface’s internal queue. If the internal queue were to grow without bound, the interface would eventually consume all available memory causing the interface to crash. In order to prevent this, the interface monitors the size of the internal queue and if this size exceeds the maximum allowed, the interface will discard new messages coming in to the interface up to the time the queue has recovered.If the size of the internal queue length causes the interface to stop reading syslog messages, the interface writes the system digital state I/O Timeout to all its tags after the internal queue is processed. The interface uses a timestamp of one second after the last syslog message was read.

This maximum queue size has a default size of 50,000, but may be adjusted by using the /mxQ=x command-line parameter. The size of the queue should be large enough to prevent transient periods of high message loads from causing messages to be lost.

Four performance counters are provided to assist in monitoring the interface load:

1. Syslog Message Queue Length: This provides the current length of the internal queue.
2. Queue Percentage Used: This is the ratio of the current queue size to the maximum queue size (mxQ), expressed as a percentage.
3. Syslog Message Process Rate: This is the approximated rate in messages/minute at which the interface processes messages. The rate the interface processes messages depends on the number of points in the interface, the type and complexity of the filter expressions in each point, the number of messages that require writing values to PI and the CPU load of the interface machine. If this rate is close to the rate messages are being sent to the syslog port, then Interface performance may be a problem. Note: This rate is approximate and may decrease as the message rate increases.
4. Syslog Message Rate: This is the rate syslog messages are being read by the interface in messages/minute.

Syslog Format and Contents

A syslog packet is a string of printable and non-printable ASCII characters. The total length of the packet must be 1024 bytes or less. Typicallya syslog packet contains three discernable parts:

• PRI (Facility and Severity)
• HEADER
• MSG

It is recommended that a syslog packet have all three parts. But there are no set requirements on the contents of the syslog packet as it is originally sent from a device. For example a syslog packet may have only the MSG part or have any part missing. The order of the parts, however, can not be interchanged.

PRI (Facility and Severity)

The PRI part starts with a leading “<”, followed by a number, which is followed by a “>”. The number contained within these angle brackets is known as the Priority value and represents both the Facility and Severity.

All syslog messages have a logging Facility and a Severity level. The logging Facility can be thought of as “where” and the Severity level can be thought of “what.” The Facilities and Severities of the messages are numerically coded with decimal values. The PRI part that contains a Priority value is included in a syslog packet and represents both the Facility and Severity. The Priority value is calculated by first multiplying the Facility number by 8 and then adding the numerical value of the Severity.

HEADER

The HEADERpart typically contains two fields called the TIMESTAMP and the HOSTNAME. The TIMESTAMP is the local time and is in the format of

Mmm dd hh:mm:ss

where:

Mmm is the English language abbreviation for the month of the year with the first character in uppercase and the other two characters in lowercase. The following are the only acceptable values: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec.

dd is the day of the month. If the day of the month is less than 10, then it must be represented as a space and then the number.

Hh:mm:ss is the local time. The hour (hh) is represented in a 24-hour format. Valid entries are between 00 and 23. The minute (mm) and second (ss) entries are between 00 and 59.

The HOSTNAME field contains either the hostname or the IP address of the originator of the message.

MSG

The MSG part usually contains some additional information of the process that generated the message, and then the text of the message. It has two fields known as the TAG field and the CONTENT field. The value in the TAG field may be the name of the program or process that generated the message. The CONTENT contains the details of the message.

As an example, a valid syslog message is as follows:

<34>Dec 18 17:58:26 mymachine su: ‘su root” failed for lonvick on /dev/pts/8

Thus

• PRI:34(Facility 4 Severity 2)
• Header:Dec 18 17:58:26 mymachine
• MSG:su: ‘su root” failed for lonvick on /dev/pts/8

But as discussed previously, the aforementioned format of the syslog messages is recommended, but not required. Therefore, different programs, processes and devices can send out syslog packets with different formats. For example, the MSG part of a System log packet sent by the PIX Firewallalways begins with a percent sign (%) and is structured as follows:

%PIX-Level-Message_number: Message_text

Syslog Interface Message Types

To facilitate the correct interpretation of each message, points in this interface can be configured to treat a syslog message as one of four categories.

PIX

Syslog messages sent by a Cisco PIX Firewall contain information about the status of connections within this firewall. Typically these messages have the form:

<PRI>TimeStampHost %PIX-Level-Message_number: Message_text

For example:

<164>Jul 16 200317:15:32OSIFirewall001 : %PIX-4-400024 IDS: 2151 Large ICMP Traffic from 10.4.1.2 to 10.2.1.1 on interface dmz

Where:

<PRI> / The PRI (facility and severity)
Timestamp / The time the message was generated
Host / The Host Name or IP address of the originating device
PIX / Identifies the message facility code for message generated by the PIX Firewall. This value is always PIX.
Level / The level reflects the severity of the condition described by the message. The lower the number, the more severe the condition.
Message_number / A unique 6-digit number that identifies the message.
Message_text / A text string describing the condition. This portion of the message sometimes includes IP addresses, port numbers or usernames.

The interface will attempt to parse out the following fields from a syslog message:

• Facility number (from the PRI)
• Severity number(from the PRI)
• TimeStamp
• Host
• Level
• MSG

Cisco IOS

Cisco devices may provide IOS messages to a syslog server. These syslog messages include messages in a standardized format (often called system error messages) and output from debug commands. Messages are of the form:

%facility-severity-mnemonic: message-text

These messages are often preceded with additional information like time and sequence-number, for example:

000013: Mar 18 14:52:10.039:%LINK-5-CHANGED: Interface Serial3/3, changed state to administratively down

The message may also be preceded by a PRI.

Syslog messages with the message component starting with %name-number-name are suitable to be considered type Cisco IOS. This does not exclude PIX type messages.

Facility Name / Identifies the message facility code,in this case LINK
Level / The level reflects the severity of the condition described by the message. The lower the number, the more severe the condition.
Mnemonic / General description of message type
Message_text / A text string describing the condition.

The interface will attempt to parse out the following fields from a syslog message

• Facility: (from PRI)
• Severity: (from PRI)
• TimeStamp: Any valid time before the first %
• Host
• Level
• MSG: The entire message after the fist %
• Facility Name: This is the facility after the first %

Syslog General

Although the Syslog standard does not impose requirements on a syslog message format, RFC 3164 – The BSD syslog protocol guidelines, provides a recommended format for syslog messages. Points of this category will treat the syslog message as if it were in this recommended format. That is, the messagewill typically be of the form: