ECS –Remote Server Upgrade Guide
January2009
Revision 1.1
Proprietary Notice
Information in this document is confidential and proprietary to itaas, Inc. Access to and use of this information is limited strictly to licensees of itaas, Inc. and is subject to the terms of a separate and appropriate written license agreement between the Licensee and itaas, Inc. The information in this document may be protected by patent applications owned by itaas, Inc.
In addition, the information in this document is subject to change without notice and does not represent a commitment on the part of itaas, Inc. The software described in this document may be used only in accordance with the terms of the license agreement. It is against the law to copy the software on any medium except as specifically allowed in the license or non-disclosure agreement. No part of this manual may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or information storage and retrieval systems, for any purpose other than the licensee’s personal use without express written permission of itaas, Inc.
All specifics of the product, its use and implementation contained in this document are provided in good faith. However, all warranties, implied or expressed, including but not limited to implied warranties of merchantability or fitness for a particular purpose are excluded.
This document is intended solely to assist the Licensee in the use of the itaas, Inc. Licensed Software Product and for no other purpose. itaas, Inc. shall not be liable, in any way under any theory of law, for any loss or damage of any kind what so ever, arising from the use of any of the information contained in this manual, or any error or omission in the information, or any incorrect use of the Software Product.
Copyright © 2007itaas, Inc. All rights reserved.
itaas is a registered trademark of itaas, Inc.
Revision History
Revision Number / Date / Description1.0 / 1/2009 / Initial version.
1.1 / 2/27/2009 / Revised
Table of Contents
1Introduction
1.1System Architecture
1.2Scope
1.3Audience
1.4Reference
2DNCS Upgrade
2.1Before Upgrade
2.2After Upgrade
3BMS Upgrade
3.1Before Upgrade
3.2After Upgrade
4nCube RT Upgrade
4.1Before Upgrade
4.2After Upgrade
5Address and Support
Table of Figures
Figure 1: System Architecture
1Introduction
The ECS tool monitors the Video on Demand (VOD) infrastructure and provides statistics and error/event information from all the various components, which can help with troubleshooting problems throughout the network. As many different vendors provide system components for the TWC VOD infrastructure, the tool collects information from the various sources and brings it into a consolidated information model that can be used to monitor system health and troubleshoot problems.
In this chapter:
- System Architecture
- Scope
- Audience
1.1System Architecture
Events are collected in near real-time. An ECS agent gathers data from the source of the event and sends it to a central event database.
Figure 1: System Architecture
1.2Scope
This document contains measures to be taken during VOD Remote Component (DNCS, BMS, Showrunner, etc.) upgrade in order to prevent disrupting of ECS processes both on the remote Components as well as the ECS server.
1.3Audience
This document is intended for system administrators responsible for upgrading the remote servers, and may be used by operators who will install and use ECS tool to monitor VOD traffic.
1.4Reference
1itaas_ECS_RemoteComponent_InstallGuide.pdf
2DNCS Upgrade
itaas, Inc. recommends the following measures be taken during upgrade of DNCS for the proper functioning of ECS tool and its support to DNCS logs.
In this section:
- Before Upgrade
- After Upgrade
2.1Before Upgrade
1Backup the ECS install directory “/opt/isaecs” on DNCS.
$ mkdir /export/home/dncs/ecsBkup
$ cd /opt
$ tar –cvf /export/home/dncs/ecsBkup/ecs.dncs.tar ./isaecs
2Save the existing “/etc/inittab” file.
$ cp /etc/inittab /export/home/dncs/ecsBkup/inittabentry.out
3Save the existing crontab.
$ cp /var/spool/cron/root /export/home/dncs/ecsBkup/rootCronEntry.out
2.2After Upgrade
1Verify ECS install directory is in place. It should have the following directory structure:
/opt/isaecs/bin
/opt/isaecs/tcpdump
/opt/isaecs/logs
/opt/isaecs/download
2If the ECS install directory is not present, extract the “ecs.dncs.tar.Z” tar ball.
Note: if ECS backup was not taken before upgrade, then refer to ECS Remote Component Install Guide for re-configuring ECS on DNCS.
$ cd /
$ tar –xzvf /export/home/dncs/ecsBkup/ecs.dncs.tar.Z
3Verify that the network interface on which the DSM-CC traffic appears has not been changed after the upgrade. If the network interface has been changed, then modify the IFACE parameter value in ‘/opt/isaecs/bin/ecsRemoteCfg.sh’script.
$ vi /opt/isaecs/bin/ecsRemoteCfg.sh
Note: Refer to section 2.1.5 of the “itaas_ECS_RemoteComponent_InstallGuide.pdf” document for details to identify the appropriate network interface.
4Verify “/etc/inittab” has the below entry for tcpdump line,
# grep tcpdump /etc/inittab
td:234:respawn:/opt/isaecs/bin/tcpdumpCmd >/opt/isaecs/tcpdump/script.out 2&1
5If the above line is missing, execute the commands to add this line, and start tcpdump process.
Note: Remember to make a copy of the inittab file prior to performing any of the following steps. You must source the root permissions (‘su –‘when changing to user root).
This entry in ‘/etc/inittab’ file is required to ensure that in case this command gets killed due to some reason, it will immediately get re-launched automatically and we will NOT lose any tcpdump packets.
# cat /opt/isaecs/install/inittabTcpdumpLine >/etc/inittab
# grep tcp /etc/inittab
Note: If the DNCS system is a V890 server and is running Solaris 10, then the above tcpdump line in the inittab needs to be modified as below,
xtd:234:respawn:/opt/isaecs/bin/tcpdumpCmd >/opt/isaecs/tcpdump/script.out 2&1
This is because the inittab entry begining with a ‘t’ will not allow a telnet session to the DNCS remotely.
# init q
The above command reads the “/etc/inittab” file, and will start the new process because of the respawn keyword.
Note: If the DSMCC listening network interfacewas changed after upgrade, then the old tcpdump process needs to be killed and restarted after the step 4 given above is executed.
# ps –ef |grep tcpdump
Issue the kill command given below to disable the old tcpdump process. Replace the ‘tcpPid’ with the process id generated by the output of the above ‘ps –ef’ command.
# kill -9 tcpPid
# init q// To restart tcpdump process
6Verify that tcpdump logs are being produced under “/opt/isaecs/tcpdump” folder.
7Verify that separate DSM, DRM and QAM are being produced after the upgrade.
8Verify that crontab has entry for “/opt/isaecs/bin/rsync_cronjob” and “/opt/isaecs/bin/logClean” processes. In addition, verify that these processes have executable permissions.
# crontab –l
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/isaecs/bin/rsync_cronjob
20 2 * * * /opt/isaecs/bin/logClean
Note: If the DNCS system is a V890 server and is running Solaris 10, then the cronjob ‘rsync_cronjob’ needs to be replaced by ‘rsyncDNCS’. This is because the system cron does not execute the softlink file. Edit the file ‘/var/spool/cron/crontabs/root’ to make changes to the crontab.
9Check the current “/opt/isaecs”logs/rsync*.log” for error messages to verify that all logs (tcpdump,dsm,drm,qam) are being rsynced over to the ECS server.
Note: In the event that the ECS agent is wiped-out from DNCS during the upgrade, refer to ‘itaas_ECS_RemoteComponent_InstallGuide.pdf’ to perform re-installation of ECS agent on the DNCS.
3BMS Upgrade
itaas, Inc. recommends the following measures be taken during upgrade of BMS (on which Central logger runs) server for the proper functioning of ECS tool and its support to BMS logs.
In this section:
- Before Upgrade
- After Upgrade
3.1Before Upgrade
1Backup the ECS install directory “/opt/isaecs” on BMS.
$ mkdir /tmp/ecsBkup
$ cd /opt
$ tar –cvf /tmp/ecsBkup/ecs.bms.tar ./isaecs
2Save the existing crontab.
$ cp /var/spool/cron/root /tmp/ecsBkup/rootCronEntry.out
3.2After Upgrade
1Verify ECS install directory is in place. It should have the following directory structure:
/opt/isaecs/bin
/opt/isaecs/logs
/opt/isaecs/download
/opt/isaecs/install
2If the ECS install directory is not present, extract the “ecs.bms.tar” tar ball.
Note: if ECS backup was not taken before upgrade, then refer to ECS Remote Component Install Guide for re-configuring ECS on BMS server.
$ cd /
$ tar –xvf /tmp/ecsBkup/ecs.bms.tar
3Verify that the crontab has entry for “/opt/isaecs/bin/rsync_cronjob” and “/opt/isaecs/bin/logClean” processes. In addition, verify that these processes have executable permissions.
Note: In the event of new BMS server, verify that the root crontab entries for ECS processes are the same as present in the previous server. Compare the cronjobs from contents of ‘/tmp/ecsBkup/rootCronEntry.out’ file stored on the previous server.
# crontab –l
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/isaecs/bin/rsync_cronjob
20 2 * * * /opt/isaecs/bin/logClean
4Check that post BMS upgrade, ‘n2bb.log’ file with current log messages is present under "/usr/local/n2bb/log” directory.
5Check the current “/opt/isaecs”logs/rsync*.log” for error messages to verify that ‘n2bb’ logs are being rsynced over to the ECS server.
Note: In the event that the ECS agent is wiped-out from BMS during the upgrade, refer to ‘itaas_ECS_RemoteComponent_InstallGuide.pdf’ to perform re-installation of ECS agent on the BMS.
4nCube RT Upgrade
itaas, Inc. recommends the following measures be taken during upgrade of nCube RT server for the proper functioning of ECS tool and its support to C-COR logs.
In this section:
- Before Upgrade
- After Upgrade
4.1Before Upgrade
1Backup the ECS install directory “/opt/isaecs” on nCube RT host server.
$ mkdir /tmp/ecsBkup
$ cd /opt
$ tar –cvf /tmp/ecsBkup/ecs.nCubeRT.tar ./isaecs
2Save the existing crontab.
$ cp /var/spool/cron/root /tmp/ecsBkup/rootCronEntry.out
4.2After Upgrade
1Verify ECS install directory is in place. It should have the following directory structure:
/opt/isaecs/bin
/opt/isaecs/logs
/opt/isaecs/download
/opt/isaecs/install
2If the ECS install directory is not present, extract the “ecs.nCubeRT.tar” tar ball.
Note: if ECS backup was not taken before upgrade, then refer to ECS Remote Component Install Guide for re-configuring ECS on nCube RT host server.
$ cd /
$ tar –xvf /tmp/ecsBkup/ecs.nCubeRT.tar
3Verify that the crontab has entry for “/opt/isaecs/bin/rsync_cronjob” and “/opt/isaecs/bin/logClean” processes. In addition, verify that these processes have executable permissions.
Note: in the event of new RT host server, verify that the root crontab entries for ECS processes are the same as the previous server. Compare the cronjobs from contents of ‘/tmp/ecsBkup/rootCronEntry.out’ file stored on the previous server.
# crontab –l
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/isaecs/bin/rsync_cronjob
20 2 * * * /opt/isaecs/bin/logClean
4Check the current “/opt/isaecs”logs/rsync*.log” for error messages to verify that RT host logs are being rsynced over to the ECS server.
Note: In the event that the ECS agent is wiped-out from RT host during the upgrade, refer to ‘itaas_ECS_RemoteComponent_InstallGuide.pdf’ to perform re-installation of ECS agent on the nCube RT host server.
5Address and Support
For questions about re-configuring the ECS agent on upgraded Remote servers that were not answered here, you can contact the itaas ECS Support team at:
Email:
Phone: 770-368-4024(Between 8.30 a.m. to 5.30 p.m. EST Monday to Friday)
For support after business hours, please contact 1-888-Go-itaas (1-888-464-8227).
Our office address:
itaas Inc.
11695 John's Creek Parkway, Suite 300,
Duluth, GA30097
Tel: 770-368-4024
Fax: 770-246-0616
Email:
Website:
- 1 -itaas, Inc., Confidential and Proprietary