BB Installation Guide s2

SAP HANA smart data streaming 1.0
June 2015
EnglishEnglish
SAP Smart Data Streaming Deployment for Stream Intelligence (EB9)
SAP SE
Dietmar-Hopp-Allee 16
69190 Walldorf
Germany / Building Block Configuration Guide

SAP Best Practices SAP SDS Deployment for Stream Intelligence (EB9): Configuration Guide

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see http://global.sap.com/corporate-en/legal/copyright/index.epx#trademark for additional trademark information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Revision / Change Date / Description
0 / June 6, 2015 / Initial

Icons

Icon / Meaning
/ Caution
/ Example
/ Note
/ Recommendation
/ Syntax

Typographic Conventions

Type Style / Description
Example text / Words or characters that appear on the screen. These include field names, screen titles, pushbuttons as well as menu names, paths and options.
Cross-references to other documentation.
Example text / Emphasized words or phrases in body text, titles of graphics and tables.
EXAMPLE TEXT / Names of elements in the system. These include report names, program names, transaction codes, table names, and individual key words of a programming language, when surrounded by body text, for example, SELECT and INCLUDE.
Example text / Screen output. This includes file and directory names and their paths, messages, source code, names of variables and parameters as well as names of installation, upgrade and database tools.
EXAMPLE TEXT / Keys on the keyboard, for example, function keys (such as F2) or the ENTER key.
Example text / Exact user entry. These are words or characters that you enter in the system exactly as they appear in the documentation.
<Example text> / Variable user entry. Pointed brackets indicate that you replace these words and characters with appropriate entries.

Content

SAP Event Stream Processor Deployment for Stream Intelligence: Configuration Guide 6

1 Purpose 6

2 Preparation 6

2.1 Prerequisites 6

3 Configuration 7

3.1 Configure SDS Users 7

3.2 Installing solution SAP SDS Components 7

3.2.1 Confirming the BDI SDS Component Install 8

3.3 Configuring the Bootstrap properties 8

3.3.1 Controlling Bootstrap with the INI file 9

3.3.2 Testing the Bootstrap Script 10

3.4 Working With the Log File Adapters 13

3.4.1 Running the Log File Adapters 14

3.5 Mapping of Log File Formats and SAP HANA Table 14

3.6 Working With the SAP SDS Source Code 17

3.7 Customizing Log File Processing 18

3.7.1 Change the Properties File 18

3.7.2 Change the SPS project code 20

4 Dynamic Loading 23

4.1 Configuration 23

4.1.1 System Tables 23

4.1.2 BDI_CONFIG_PARAMETERS Options 24

4.1.3 Initial Configuration Scripts 24

4.1.4 Starting Daemon 26

5 Additional Resources 26

5.1 SAP Smart Data Streaming (SAP SDS) 26

SAP Best Practices SAP SDS Deployment for Stream Intelligence (EB9): Configuration Guide

SAP Event Stream Processor Deployment for Stream Intelligence: Configuration Guide

1 Purpose

The purpose of this document is to describe the general configuration steps required to manually set up the SAP HANA® smart data streaming (SAP SDS) configuration for Stream Intelligence within the SAP HANA landscape that has already been installed using the corresponding installation or configuration guides for installation.

This installation does not support multi-tenancy databases.

2 Preparation

2.1 Prerequisites

Before you start installing this scenario, you must install the prerequisite building blocks. For more information, see the Latest Information and Configuration Guide: Getting Started on the SAP Service Marketplace for this solution, http://service.sap.com/rds-bdi .

The following components are installed and configured prior to starting the Stream solution installation. These components are as follows:

1. SAP HANA smart data streaming (SDS)

Or optionally, SAP® Event Stream Processing Software (SAP ESP)

2. UNIX ODBC Driver

3. SAP IQ (Optional)

4. Hadoop (Optional).

The ODBC driver is expected to be installed on the host machine for the SDS and/or SAP ESP servers. If, optionally, nearline storage will be used in the implementation, the client drivers for SAP IQ and/or Hadoop are required to be installed on this host machine also.

3 Configuration

Setting up SAP SDS technical components for the solution require the following high-level processes:

1. Install and configure SAP Stream software

2. Prepare the software bootstrap environment

The following sections contain UNIX/Linux operations. We recommend that an experienced UNIX user performs the configuration steps.

Many commands will start with, or contain the characters “dot-slash” ./ This references specific files relative to the current directory, as opposed to using the programs available through the PATH environment variable. It should always be included where indicated in the configuration instructions.

It is assumed that the SAP HANA client is already installed.

3.1 Configure SDS Users

The SDS user “SYS_STREAMING” needs to be added prior to running your bootstrap process. To configure this user, the following commands need to be executed on the SDS/ESP servers:

./streamingclusteradmin --uri=esps://<sds hostname>:<sds port> --username=SYSTEM --password=<password>

grant permission all to user SYS_STREAMING

quit

./ streamingclusteradmin --uri=esps://<sds hostname>:<sds port> --username=SYSTEM --password=<password>

grant permission all to user SYS_STREAMING

quit

3.2 Installing solution SAP SDS Components

The SAP HANA smart data streaming (SDS) environment for the rapid-deployment solution consists of a group of files that support the operation of the system. These files have been developed to be configured automatically based on local settings for server IDs, logon credentials and SAP HANA server environment.

SDS has two logical workspace definitions, one for the SAP HANA studio, where source code is developed, and the other for the deployed server, where runtime code is staged to be loaded into the SDS node.

This solution uses the same physical base directory for all its processing. This includes the storage of source code, executable code, configuration, and adapter management.

The installation of the provided solution components requires extracting the contents of a zip file to the SDS projects base directory as follows:

cd $STREAMING_HOME/cluster/projects

tar –xf stream_rds_bdi.tar

3.2.1 Confirming the BDI SDS Component Install

Once the unzip process has completed, check the directory structure with this command:

ls –l $STREAMING_HOME/cluster/projects

Verify that the directory listing includes the following folders:

· rds_bdi_stream

· repository

· run_stream_adapter

No further configuration activity is required for this step.

3.3 Configuring the Bootstrap properties

The core to the deployment of the solution SAP HANA smart data streaming system is the bootstrap script, which customizes the software to use the local servers, user credentials and other user preferences. You manage by setting the values in the SAP HANA Attribute View named <AT_BDI_CONFIG>. This script can be executed as many times as needed when the configuration values have changed. The script removes all the deployed code related to the STREAM application and replaces with the updated code. This also includes the ODBC definitions for HANA (ESP_HANA_STREAM) and IQ (ESP_IQ_STREAM).

The script <bootstream.py> is written in Python and requires Python to be installed prior to running this script. If SDS and SAP HANA are installed on the same machine, Python is available in the folder “${DIR_EXECUTABLE}/Python/bin/”. If SDS or SAP ESP are on a separate host machine and Python is not installed with the applications, it may be accessed in the system folder “/usr/bin/python”. If not, request your system administrators to install Python on the host machine before proceeding.

After Python is install, update the <bootstream.py> with the current location of Python. The first line of the script <bootstream.py> reads: “#!/usr/sap/HDB/HDB00/exe/Python/bin/python”

Update the line to the new path of the Python application on the server.

The <bootstream.py> script takes values from <AT_BDI_CONFIG> and applies to the following system and application files:

· odbc.ini – This file manages the connection properties to the database servers

· services.xml – This SDS staging file is used to load data service definitions into the cluster database

· IQ DDL files – If SAP IQ is configured for this system, these DDL files are processed to ensure that the SAP IQ environment is set up correctly.

· STREAM SDS Source – If the STREAM application is being used, its source code is configured, loaded into the SDS node and started.

Configuring the bootstrap properties involves opening the stream.ini> file in the vi editor, and checking that it contains the correct values for your system.

Most settings are already correct. The bootstrap file allows the use of environment variables, so the lines that begin with <$STREAMING_HOME will translate to the correct SDS base directory for your system.

The section “ODBC_INI” must be modified to update the destination of the INI file based on the deployment target. If deploying to SDS on an SAP HANA host machine, the deployment “Destination” for the ODBC file is “/usr/sap/HDB/home/.odbc.ini”. If deploying to an SAP ESP server on a standalone machine, the deployment “Destination” is “/usr/local/etc/odbc.ini”

The section that must be configured is named <[CONFIG_TABLE]>. It contains the following properties:

· SERVER=<server ID> – This is the domain name or IP address of the SAP HANA server

· PORT=3xx15 – This is the SAP HANA port number. The “xx” value is replaced by the SAP HANA instance number.

· ATTRIBUTE VIEW=“sap.rds-bdi.stream.config/AT_BDI_CONFIG“ – This is the default location for the configuration Attribute View. Confirm that this is the correct location and path for your system.

Lines starting with the hash symbol (#) in the stream.ini> file are treated as a comment line.

3.3.1 Controlling Bootstrap with the INI file

The bootstrap script is designed to configure the deliverable package for the Stream solution. The configuration of this deliverable is controlled with the stream.ini file that provides the bootstrap with the necessary information on how to locate and configure various system components. The following section information will require updating. The entries for CONFIG_TABLE/SERVER and CONFIG_TABLE/ PORT will need to be updated to the match that of the HANA server that is hosting the repository information. The scripts esp_admin.sh and sds_admin.sh will also updating, but are dependent on the implementation platform. If SDS is being used, only the script sds_admin.sh needs to be updated with the information required to connect to the SDS server. Correspondingly, is ESP is being used, then the script esp_admin.sh needs to be updated with the information to connect to the ESP server.

"SAP_RDS_BDI_STREAM"."sap.rds-bdi.stream.model::config.BDI_CONFIG_PARAMETERS"

3.3.2 Testing the Bootstrap Script

The only way to verify that the bootstrap properties are set up correctly is to run the script as follows:

1. Make sure you are in the repository directory.

cd $STREAMING_HOME/cluster/projects/repository

2. If using SDS, confirm that the SDS is running.

./sds_admin.sh

>quit

3. If using ESP, confirm that the SAP ESP is running.

./esp_admin.sh

>quit

4. Execute the bootstrap script

./bootstream.py <HANA User> <HANA Password>

Where:

<HANA User> – is the SAP HANA logon name

<HANA Password> – is the SAP HANA logon password

The script produces output similar to the following. Review your output carefully for errors, and make the appropriate corrections to your stream.ini> file as needed.

Base directory names in the following output may be different from your system.

*****************************************************

* Starting the Bootstrap process

*****************************************************

ODBC_INI-TEMPLATE

*****************************************************

[convertFile] processing /usr/local/etc/odbc.ini

[processFile] File processed successfully: /usr/local/etc/odbc.ini

Ensure that the process is successful and the odbc.ini file is in /usr/local/etc

*****************************************************

SERVICE_XML-TEMPLATE

*****************************************************

[convertFile] processing /data/sybase/ESP-5_1/cluster/projects/esp1/repository/service_new.xml

[processFile] File processed successfully: /data/sybase/ESP-5_1/cluster/projects/esp1/repository/service_new.xml

Executing ESP Services file.

[processServices] Removing Service named: streamhanaservice

[done]

[processServices] Removing Service named: streamiqservice

[done]