Universal File Loader
Interface to the PI System
Version 2.1.0.0
Rev D
69
Universal File Loader Interface to the PI System 69
How to Contact Us
Phone / (510) 297-5800 (main number)(510) 297-5828 (technical support)
Fax / (510) 357-8136
E-mail /
World Wide Web / http://www.osisoft.com
Mail / OSIsoft
P.O. Box 727
San Leandro, CA 94577-0427
USA
OSI Software GmbH
Hauptstrabe 30
D-63674 Altenstadt 1
Deutschland / OSI Software, Ltd
P O Box 8256
Symonds Street
Auckland 1035 New Zealand
OSI Software, Asia Pte Ltd
152 Beach Road
#09-06 Gateway East
Singapore, 189721
Unpublished -- rights reserved under the copyright laws of the United States.
RESTRICTED RIGHTS LEGEND
Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c)(1)(ii)
of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013
Trademark statement—PI is a registered trademark of OSI Software, Inc. Microsoft Windows, Microsoft Windows for Workgroups, and Microsoft NT are registered trademarks of Microsoft Corporation. Solaris is a registered trademark of Sun Microsystems. HPUX is a registered trademark of Hewlett Packard Corp.. IBM AIX RS/6000 is a registered trademark of the IBM Corporation. DUX, DEC VAX and DEC Alpha are registered trademarks of the Digital Equipment Corporation.
PI_UFL.doc
Ó 2002, 2003, 2004 OSIsoft, Inc. All rights reserved
777 Davis Street, Suite 250, San Leandro, CA 94577
69
Universal File Loader Interface to the PI System 69
Table of Contents
Introduction 1
Reference Manuals 1
Supported Features 1
Principles of Operation 3
Startup 3
Runtime Operation 3
Installation Checklist 5
Interface Installation 7
Naming Conventions and Requirements 7
Microsoft DLLs 8
Interface Directories 8
Interface Installation Procedure 8
Installing the Interface as an NT Service 9
Digital States 11
PointSource 13
PI Point Configuration 15
Point Attributes 15
Output Points 16
Configuration File 19
General 19
Comments 19
Line Continuation 20
File Settings: [FILE] 20
Message Settings: [MSG] 20
Field Settings: [FIELD] 21
[MSG(n)] Message Structure Definitions 25
Performance Point Configuration 35
I/O Rate Tag Configuration 37
Startup Command File 39
Command-Line Parameters 39
Sample PI_UFL.bat File 42
Interface Node Clock 45
Security 47
Starting / Stopping the Interface 49
Starting Interface as a Service 49
Stopping Interface Running as a Service 49
Buffering 51
Configuring Buffering with PI-ICU (NT-Intel) 51
Configuring Buffering Manually 54
Example piclient.ini File 55
Appendix A: Error and Informational Messages 57
Message Logs 57
Messages 57
Appendix B: CSV (Comma Delimited) Data Files 61
Example 61
Appendix C: XML Document Files 63
Example 63
Appendix D: ASCII Data File Examples 67
Example 67
Revision History 71
69
Universal File Loader Interface to the PI System 69
Introduction
This document describes OSIsoft’s Universal File Loader (PI_UFL) interface to the PI system. The PI_UFL interface parses data from various formats ASCII files and stores the data into PI.
The Interface processes files that match user defined criteria, starting with the oldest file. A configuration file controls how the interface parses data files and stores data to PI. The content of each data file is read line by line. Each line is checked to see whether it matches one of several sets of criteria. If a line matches a set of criteria, data is extracted from the line based on these criteria and the results are stored in variables. The contents of these variables can be manipulated, and on request a set of values can be sent to PI. To operate the interface effectively, the user must thoroughly read the section of this manual concerning the syntax of the configuration file.
The Interface runs on Intel NT machines running NT 4.0 SR 4 or higher. This machine may be either a PI home or PI Interface node.
This document contains the following topics:
· a brief design overview
· a section concerning the details of creating points to be scanned by this interface
· a section concerning the configuration file
· a section concerning the command line parameters
· an overview of the software configuration
· details of operation and installation.
Reference Manuals
OSIsoft
· PI Data Archive Manual
· PI-API Installation Instructions
Supported Features
Feature / Support /Part Number / PI-IN-OS-UFL-NTI
Platforms / Windows NT 4.0 / 2000 / XP
PI Point Types / PI 2: real / digital / integer
PI 3: int16 / int32 / float16 / float32 / float64 / digital / string
Sub-Second Timestamps / Yes
Sub-Second Scan Classes / No
Automatically Incorporates PIPoint Attribute Changes / Yes
Exception Reporting / Yes
Outputs from PI / No
Inputs to PI / Scan-Based
Maximum Point Count / Unlimited
Uses PI-SDK / No
PINet to PI 3 String Support / No
* Source of Timestamps / Timestamps are read from the input file.
* History Recovery / Yes
Failover / No
UniInt-Based / No
Vendor Software Required on PI-API / PINet Node / Not applicable
Vendor Software Required on Foreign Device / Not applicable
Vendor Hardware Required / Not applicable
Additional PI Software Included with Interface / Not applicable
Device Point Types / Not applicable
* See paragraphs below for further explanation.
Source of Timestamps
Timestamps are read from the input file or, when not specified, current server time is used.
History Recovery
History recovery is automatically included with a file-based interface. After the interface has been down for some reason, all files containing data will be processed. As long as these data files are not deleted, they will be processed by the interface the 1st time the interface is run, no matter how much data is stored in these files, no matter how long the interface has been down.
69
Universal File Loader Interface to the PI System 69
Principles of Operation
Startup
At startup, the PI_UFL interface reads the configuration file. The path of the configuration file is specified as an argument on the command line. The configuration file tells the PI_UFL interface how to interpret data from the input files.
The interface performs a series of syntax checks on the configuration or message definition file and then compiles the contents of the file. If errors are found in the configuration file, messages are written to the output file and the interface halts.
Once the configuration file has been read, the interface reads the PI point database according to the specifications on the startup command line.
· If /ps is specified, all PI points with that PointSource are loaded into the interface. If the configuration file specifies that the value in the message should be sent to the PI point with the given InstrumentTag, only the list of tags loaded at startup are searched. If no matching InstrumentTag is found, the data is not sent to PI. If the Tag name is used, the interface first searches through the list of tags loaded at startup with the given point source. If the tag is not found in the internal list, the entire PI point database is searched.
· If /tm is specified, all PI points with a tag name that fits the tag mask are loaded into the interface. If the configuration file specifies that the value in the message should be sent to the PI point with the given InstrumentTag, only the list of tags loaded at startup are searched. If no matching InstrumentTag is found, the data is not sent to PI. If the Tag name is used, the interface first searches through the list of tags loaded at startup, which match the mask. If the tag is not found in the internal list, the entire PI point database is searched.
· The optional /tm and /ps parameters are mutually exclusive. If neither is specified on the command line, no PI points are loaded at startup. In this case the Tag name must be used in the data file. For optimization the interface will remember a tag that was once accessed, but that is not the same as the internal tag list. Without /tm or /ps InstrumentTag will never find a match.
If all configuration steps are completed successfully, the interface continues with Runtime operation.
Runtime Operation
The PI_UFL interface checks at regular time intervals whether new input files have been created. The scan frequency is specified as an argument, /f=hh:mm:ss,hh,mm,ss, on the command line. For more information on command-line parameters, see section Command-line Parameters.
All files that match the input file mask are treated as input files. The input file mask is specified on the command line.
· The order in which the files are read is based on the file creation time with the oldest files read first.
· Each input file is read, line per line. The PI_UFL interface compares each line read from the input file with the message declarations in the configuration file, starting with the message declared first in the file. As soon as a FILTER condition of a message definition is found that matches the line, the statements in the message definition are executed and the rest of the message definitions are skipped.
Note: If the line from the input file does not match any filter definition, it is skipped, no error is reported.
· After an input file is processed, it is renamed with an extension indicating successful processing. If the file contained an error, it is renamed with a different extension, indicating an error while processing. Both extensions can be specified on the command line.
· After a given time period, files that have been processed without error are deleted. This purge interval is specified by /pu on the command line. Files that contained errors are NOT purged.
69
Universal File Loader Interface to the PI System 69
Installation Checklist
For those users who are familiar with running PI data collection interface programs, this checklist helps you get the PI_UFL interface running. If you are not familiar with PI interfaces, you should return to this section after reading the rest of the manual in detail.
1. Verify that PI-API has been installed; if not, install the PI-API
2. Install the PI_UFL interface.
3. Create digital states.
4. Choose a point source. If PI 2 home node, create the point source.
5. Configure PI points.
6. Create a configuration file.
7. Edit startup command file.
8. Set interface node clock.
9. Set up security.
10. Test the interface by running it interactively with a high debug level or the /test flag set.
11. Start the interface as a service, but without buffering.
12. Verify data.
13. Stop interface, start buffering, start interface.
69
Universal File Loader Interface to the PI System 69
Interface Installation
OSIsoft recommends that interfaces be installed on PI Interface nodes instead of directly on the PIServer node. A PI-API node is any node other than the PI Server node where the PIApplication Programming Interface (PI-API) has been installed (see the PIAPIInstallation Instructions manual). With this approach, the PI Server need not compete with interfaces for the machine’s resources. The primary function of the PIServer is to archive data and to service clients that request data.
After the interface has been installed and tested, Bufserv should be enabled on the PI Interfacenode (once again, see the PI-API Installation Instructions manual). Bufserv is distributed with the PI-API. It is a utility program that provides the capability to store and forward events to a PI Server, allowing continuous data collection when communication to the PI Server is lost. Communication will be lost when there are network problems or when the PI Server is shut down for maintenance, upgrades, backups, or unexpected failures.
In most cases, interfaces on PI Interface nodes should be installed as automatic services. Services keep running after the user logs off. Automatic services automatically restart when the computer is restarted, which is useful in the event of a power failure.
The guidelines are different if an interface is installed on the PI Server node. In this case, the typical procedure is to install the PI Server as an automatic service and interfaces as manual services that are launched by site-specific command files when the PI Server is started. Interfaces that are started as manual services are also stopped in conjunction with the PI Server by site-specific command files. This typical scenario assumes that Bufserv is not enabled on the PI Server node. Bufserv can be enabled on the PI Server node so that interfaces on the PI Server node do not need to be started and stopped in conjunction with PI, but it is not standard practice to enable buffering on the PI Server node. See the UniInt End User Document for special procedural information.
Naming Conventions and Requirements
In the installation procedure below, it is assumed that the name of the interface executable is PI_UFL.exe and that the startup command file is called PI_UFL.bat.
It is customary for the user to rename the executable and the startup command file when multiple copies of the interface are run. For example, one would typically use PI_UFL1.exe and PI_UFL1.bat for interface number 1, PI_UFL2.exe and PI_UFL2.bat for interface number 2, and so on. When an interface is run as a service, the executable and the command file must have the same root name because the service looks for its command-line arguments in a file that has the same root name.
Interface Directories
The PIHOME Directory Tree
The PIHOME directory tree is defined by the PIHOME entry in the pipc.iniconfiguration file. This pipc.ini file is an ASCII text file, which is located in the WinNT directory. A typical pipc.ini file contains the following lines:
[PIPC]
PIHOME=c:\pipc
The above lines define the \pipc directory as the root of the PIHOME directory tree on the C: drive. OSIsoft recommends using \pipc as the root directory name. The PIHOMEdirectory does not need to be on the C: drive.
Interface Installation Directory
Place all copies of the interface into a single directory. The suggested directory is:
PIHOME\interfaces\PI_UFL\
Replace PIHOME with the corresponding entry in the pipc.ini file.
Interface Installation Procedure
In the installation procedure below, assume that interface number 1 is being installed and that all copies of the interface will be installed in the same directory.