Rational Data Collection Service January Tech Preview Installation Process

Rational Data Collection Component – Installation Process

**********ONLY CLM V 5.0, 4.0.5, 4.0.4 and 4.0.3 are supported in this version for point product data collection. Note that DCC still needs to be registered with a 5.0 Jazz Team Server and the target warehouse also needs to be at the 5.0 version ******

Pre-Installation Considerations for the Test Environment

Scenario1: Collect only Test Data in DCC

Create a new CLM 5.0 warehouse or migrate an existing test warehouse to CLM 5.0 using repotools.

Scenario2: Collect Production Data in DCC

Take a copy of the production warehouse to your test environment.

Migrate it to CLM 5.0 using repotools.

Disable any ETL running or scheduled to run for which the target warehouse is the same one as the one you will be configuring in the Rational Data Collection Component.

Pre-Requisites

Hardware Requirements

Database server

• 64 bit server
• 8 CPU cores
• 16G RAM

DCC Server

• 64 bit server
• 8 CPU cores
• 16G RAM

Data Collection Component Server

Database Server for the Warehouse

RDBMS: 64-bit as documented in Rational Insight 1.1.1.3 and CLM 5.0.

For this milestone only DB2 9.7 64-bit, Oracle 11G R2 and SQL Server 2008 R2 will be supported.

DCC Server

• Application server: WebSphere 8.0.5 and 8.5.5 64-bit, Tomcat 64-bit - version provided with CLM 5.0
• Operating System:
• Windows 2008 R2 (64-bit)
• RHEL 5.6 64-bit
• . Jazz Team Server version 5.0 installed
• Configure with Tomcat or WebSphere Application Server
• Depending to on the scenario use an existing test RRDI/CLM warehouse (migrate it to 5.0 using repotools) or create a new test warehouse.

Downloading the Software

You can install the Data Collection Component as part of the Rational Reporting for Development Intelligence package.

1. In your browser, navigate to the Rational Reporting for Development Intelligence downloads page.
2. Select the Rational Reporting for Development Intelligence 5.0 M3 Milestone.
3. Switch to the All Downloads tab.
4. To install using the IBM Installation Manager repository, download the Installation Manager repository location.

Installing the Software

Additional Database Table

The warehouse needs an additional table to be created to be used by DCC at runtime. The new table resides under the CONFIG schema. An account with the appropriate permissions needs to be used to create the table. Execute the appropriate statement according to the DBMS you are using.

DB2

CREATE TABLE CONFIG.DCS_ETL_INFO

( RESGROUP_URL VARCHAR (2048) NOT NULL ,

RESGROUP_ID VARCHAR (36) NOT NULL ,

RESOURCE_URL VARCHAR (2048) NOT NULL ,

RESOURCE_ID VARCHAR (36) NOT NULL ,

MODIFIEDSINCE TIMESTAMP NOT NULL WITH DEFAULT CURRENT TIMESTAMP,

LOADTYPE CHARACTER (1) NOT NULL WITH DEFAULT 'D' ,

CONSTRAINT DCSETLINFO_KEY PRIMARY KEY ( RESGROUP_ID, RESOURCE_ID) )

IN VNF_32K INDEX IN VNF_IDX;

Oracle

CREATE TABLE CONFIG.DCS_ETL_INFO

( RESGROUP_URL VARCHAR (2048) NOT NULL ,

RESGROUP_ID VARCHAR (36) NOT NULL ,

RESOURCE_URL VARCHAR (2048) NOT NULL ,

RESOURCE_ID VARCHAR (36) NOT NULL ,

MODIFIEDSINCE TIMESTAMP DEFAULT TO_DATE('1899-12-31','YYYY-MM-DD') NOT NULL ,

LOADTYPE CHAR (1) DEFAULT 'D' NOT NULL )

TABLESPACE VNF_32K ;

ALTER TABLE CONFIG.DCS_ETL_INFO ADD CONSTRAINT DCSETLINFO_KEY PRIMARY KEY (RESGROUP_ID, RESOURCE_ID) USING INDEX TABLESPACE VNF_IDX;

SQL Server

CREATE TABLE [CONFIG].DCS_ETL_INFO

( RESGROUP_URL NVARCHAR (2048) NOT NULL ,

RESGROUP_ID NVARCHAR (36) NOT NULL ,

RESOURCE_URL NVARCHAR (2048) NOT NULL ,

RESOURCE_ID NVARCHAR (36) NOT NULL ,

MODIFIEDSINCE DATETIME DEFAULT '1899-12-31 23:59:59.999' NOT NULL,

LOADTYPE NCHAR (1) DEFAULT 'D' NOT NULL,

CONSTRAINT DCSETLINFO_KEY PRIMARY KEY ( RESGROUP_ID, RESOURCE_ID) )

ON VNF_32K

GO

DB2 Parameter Changes

Due to the parallel processing that DCC does with data, the database at a given collection will be very active. Below are some guidelines for some of the parameters. Adjust these settings accordingly depending on your data load initially and over time.

The warehouse database must have the MAXAPPLS increased to allow for concurrent connections in DCC to process data if it is not set to AUTOMATIC. Increase the value to 300.

UPDATE DB CFG FOR <DBALIAS> USING MAXAPPLS 300

This means that the lock list will need to be increased as well if it is not set to AUTOMATIC

UPDATE DB CFG FOR <DBALIAS> USING LOCKLIST 20000

The transaction logs will also grow as data is processed in parallel. Increase the LOGFILSIZ to 20000

UPDATE DB CFG FOR <DBALIAS> USING LOGFILSIZ 20000

Increase the number of primary and secondary transactions log files.

UPDATE DB CFG FOR <DBALIAS> USING LOGPRIMARY 50

UPDATE DB CFG FOR <DBALIAS> USING LOGSECOND 200

Oracle Parameter Changes

The warehouse database needs to have the PROCESSES configuration increased to 3000.

Depending on how you start up the Oracle database, you will either have to change your PFILE or SPFILE.

Changing the SPFILE, to do so you need to connect to the Oracle database as sysdba and issue:

ALTER SYSTEM SET PROCESSES=3000 SCOPE=SPFILE;

To change the PFILE, open it in an editor, search for the PROCESSES parameter and change its current value to 3000.

Restart the Oracle database in order for the setting to take effect.

Database Driver Locations

The Oracle or SQL Server drivers during setup will be referenced during the setup from the <DCC INSTALLATION DIR>\JazzTeamServer\server\oracle and <DCC INSTALLATION DIR>\JazzTeamServer\server\sqlserver respectively. Those directories are not there by default, therefore you need to create them and place ojdbc6.jar (oracle driver) and sqljdbc4.jar (sql server driver) in these directories respectively.

At runtime DCC will load the driver in the case of tomcat from the <DCC INSTALLATION DIR>\JazzTeamServer\server\jre\lib\ext directory or in the case of WebSphere, from the <WAS HOME>\java\jre\lib\ext directory. Therefore a copy of the drivers must be placed in there as well before installing or running DCC.

Installation Manager Steps

Installation of Rational Data Collection Component is accomplished through Installation Manager V1.7. Installation Manager must be downloaded and installed separately from Jazz.net:

Start Installation Manager and select File > Preferences > Repositories.

On the Repositories preferences page, click Add Repository. The Add Repository dialog opens. Click Browse to select repository.config from the temporary directory where you extracted the archive file. Click Open to select the file. Click OK to add the repository to the repositories list.

Select the Data Collection Component repository location, and then click OK to close the Preferences dialog.

Click Install to start installing DCC.

Select IBM Rational Reporting Data Collection Component and click Next to begin the installation.

Read and accept the license agreement. When you get to the package and software installation screen, make sure that you select to install the 64-bit version of the software. You can specify an alternate location for the installation directory if you choose to. Click on Next.

Click Next, and Next again. Review the information before moving on.

The installation progress will be shown; you can monitor it from there. Once the installation completes, click Finish.

The DCC server is installed to <RRDI Install Dir>/dcc, for example C:/Program Files/IBM/RRDI/dcc.

Configuring DCC with Tomcat

The DCC server will by default occupy ports 11080 for HTTP and 11443 for HTTPS.

If you need to change the ports, backup your server.xml file located in

<DCC Installation directory>\JazzTeamServer\server\tomcat\conf and edit the appropriate port numbers in the original server.xml file.

To start the application server, go to Start > Programs > IBM Rational Reporting > Start the Data Collection Component Server, or you can start the server from

<DCC Installation Directory>\JazzTeamServer\server\server.startup.bat

Configuring DCC with WebSphere Application Server

The overall steps to deploy into WebSphere Application Server can be summarized as follows:

• Create a WebSphere application server profile
• Configure parameters and security for the profile created
• Deploy the DCC war file

Create a WebSphere Application Server Profile

The profile is created from the command line by invoking the WebSphere manageprofiles.bat batch file on Windows. The sample command (replace the paths and user password fields with the correct ones) to issue is as follows:

<WAS INSTALL DIRECTORY>\bin\manageprofiles.bat -create -nodeName %COMPUTERNAME%Node01 -adminUserName <username> -adminPassword <user password> -enableAdminSecurity true

Make sure to remember the user and password you entered for this command as you will need it to configure the application server and deploy the DCC war file.

A concrete sample command:

C:\Apps\IBM\WebSphere\AppServer\bin\manageprofiles.bat -create -nodeName %COMPUTERNAME%Node01 -adminUserName maria -adminPassword DCCRocks -enableAdminSecurity true

Stop and start the server.

Configure the Application Server

A Java python script is used to configure the application server created above. Its name and location is <DCC Installation Directory>\JazzTeamServer\server\was\clm_was_config.py

To invoke it with the correct parameters issue the following command making sure to substitute with the appropriate values. NOTE: Not all slashes are “\” some are “/” when referring to a path.

< WAS INSTALL DIRECTORY >\profiles\<application server created from the previous step>\bin\wsadmin.bat -language jython -user <username> -password <user password> -f <DCC Installation Directory>/JazzTeamServer/server/was/clm_was_config.py <DCC Installation Directory>/JazzTeamServer/server/conf

A concrete sample command (mind the slashes \ vs /):

C:\Apps\IBM\WebSphere\AppServer\profiles\AppSrv01\bin\wsadmin.bat -language jython -user maria -password DCCRocks -f C:/Apps/IBM/DCC/JazzTeamServer/server/was/clm_was_config.py C:/Apps/IBM/DCC/JazzTeamServer/server/conf

Stop and start the application server:

< WAS INSTALL DIRECTORY >\profiles\<application server created from the previous step>\bin\StopServer.bat server1

Note: You will get prompted for a user name and password when stopping. It is the same user/password as when you created the profile.

< WAS INSTALL DIRECTORY >\profiles\<application server created from the previous step>\bin\StartServer.bat server1

A concrete sample command:

C:\Apps\IBM\WebSphere\AppServer\profiles\AppSrv01\bin\StopServer.bat server1

(when prompted input user maria and password DCCRocks)

C:\Apps\IBM\WebSphere\AppServer\profiles\AppSrv01\bin\StartServer.bat server1

Deploy the DCC War File

The deployment of the war file is also accomplished via a Java python script. Its name and location is <DCC Installation Directory>\JazzTeamServer\server\was\ dcc_deploy.py

Make a copy of the file above. Open the original for editing. Replace all instances of [UID] with the user name you had chosen in the above steps. In our concrete sample we would be replacing [UID] with maria. Save the file.

To invoke it with the correct parameters issue the following command making sure to substitute with the appropriate values. NOTE: Not all slashes are “\” some are “/” when referring to a path.

< WAS INSTALL DIRECTORY >\profiles\<application server created from the previous step>\bin\wsadmin.bat -language jython –user <username> -password <user password> -f <DCC Installation Directory>/JazzTeamServer/server/was/dcc_deploy.py %COMPUTERNAME%Node01 server1 <DCC Installation Directory>/JazzTeamServer/server/tomcat/webapps/

A concrete sample command (mind the slashes \ vs /):

C:\Apps32\IBM\WebSphere\AppServer\profiles\AppSrv01\bin\wsadmin.bat -language jython -user maria -password DCCRocks -f C:/Apps/IBM/DCC/JazzTeamServer/server/was/dcc_deploy.py %COMPUTERNAME%Node01 server1 C:/Apps/IBM/DCC/JazzTeamServer/server/tomcat/webapps/

Stop and start the application server:

< WAS INSTALL DIRECTORY >\profiles\<application server created from the previous step>\bin\StopServer.bat server1

Note: You will get prompted for a user name and password when stopping. It is the same user/password as when you created the profile.

< WAS INSTALL DIRECTORY >\profiles\<application server created from the previous step>\bin\StartServer.bat server1

A concrete sample command:

C:\Apps\IBM\WebSphere\AppServer\profiles\AppSrv01\bin\StopServer.bat server1

(when prompted input user maria and password DCCRocks)

C:\Apps\IBM\WebSphere\AppServer\profiles\AppSrv01\bin\StartServer.bat server1

Confirm the Deployment and Configuration

Launch the WebSphere Administration Console for the application server that you created above:

When prompted for a user/password, use the same account information from the previous steps. In our concrete example we would be using maria and DCCRocks

Confirm that the DCC war file is deployed. It will be listed as an “Enterprise application as dcc.war.

Configuring The Data Collection Component

Setting up DCC in the Jazz Team Server

The Data Collection Component application server needs to be started as well as the Jazz Team Server 5.0 if they are not already running (see above on how to start the application server appropriate to your case).

Go to the Jazz Team Server setup page.

port>/jts/setup

Navigate through the pages until you reach the step where you can register applications. Click Add Application.

Enter the URL to the scr for the Data Collection Component. The URL has the format of :

server>:<DCC application server port>/dcc/scr

If the URL entered is correct, it will automatically fill in the “Application Instance” and the “Functional User ID”

Click Register Applications.

Click Next until you reach the configuration point for the “Rational Data Collection Component”. Login when prompted. If your application server configured for DCC is WebSphere Appication Server, then use the user/password that you used when you were configuring and deploying the war file in the previous steps. In our working sample, that would be maria/DCCRocks. If you have deployed using Tomcat, then use the standard ADMIN/ADMIN. The users will be synchronized during DCC finalization and the account used will be the one you are currently logged into the JTS to run setup.

Accept Derby as your database or set up the repository with any other DBMS and click Next.

Click Finalize Application Setup and then Next.

On the warehouse page, configure the database information for the database you set up in the test environment that is already at level 5.0. DO NOT POINT TO YOUR PRODUCTION WAREHOUSE DATABASE.

Test the connection and if it succeeds then click Next.

Click on Finalize Application Setup, then click Next for all subsequent pages until the last page where the Finish button is enabled. Then click Finish.

Configuring DCC

DISABLE ANY CLM JAVA BASED ETL OR DATA MANAGER ETL.

The URL to the DCC application is Server>:<ssl port>/dcc/web. Go to the main page. It will take a few minutes for the DCC repository to initialize in the background. Once loaded you will get a list of available collection jobs.

Once DCC has finished loading its data go to the Resource Group Configuration page to add CLM application deployments that will be processed by DCC.

Click Discover to add CLM repositories to process in DCC.

The default behavior is to list applications registered in the current Jazz Team Server that DCC is also registered with:

To register applications from a remote Jazz Team Server select the Use an external JTS to discover applications check box.

Enter the root URI to the remote Jazz Team Server and click Discover.

You will get a list of applications registered on the remote Jazz Team Server. For each application you want DCC to process, select the box next to it and click OK.

Each of the instances you selected for processing will now be listed under the individual product groups. Expand each relevant group and configure each repository. Provide the user and password to use to connect to the server to collect data when the collections run and “Test the Connection”. They should have access to all the projects in a given application, i.e JazzAdmin authority. Additionally identify the version of the software that the point product is running. i.e. 5.0, 4.0.5, 4.0.4 or 4.0.3

Expand Relational Database and configure the data warehouse’s connection information and “Test the Connection”. Do not change the version of the warehouse. Leave it at 5.0.

Once completed, click Save.

In the Delta Load Configuration page, select the load type (Full or Delta) to run. In the case of a delta load, specify the modification date the collection service should consider for the incoming data for collection for the run.

DISABLE ANY CLM JAVA BASED ETL OR DATA MANAGER ETL.

Go back to the “Data Collection Jobs” page. Note that the builds are enabled based on the resource group types registered and their version. In the example below, 4.0.5 repositories were registered hence the 4.0.3 jobs are disabled. Click Run all data warehouse collection jobs.

You can also schedule the collections to run at an interval or fixed time of day. To do so, click Schedule.

Note: ODS builds and data mart builds will not run concurrently. Do not create overlapping schedules for them. Most likely you will need to refresh the data mart at a lower frequency than the ODS. Select the type of schedule and days of the week the schedule is valid for.

Monitor the execution of the jobs in the Data Collection Job Status page.