COE Installation Quality Bulletin
** Issue 1 is Rescinded **
Installation Quality Bulletin Number: 04-01
To: All Installation Suppliers / Central Office Operations Representatives / Design Engineering Representatives / Quality Managers / Field Engineering Representatives / AMC Representatives
Initial Release Date: Friday, January 23, 2004
Revised Release Date: Friday, July 2, 2010
From: QWEST Installation Quality Assurance
Subject: DC Power “Hot Cutover” Procedures and Required Text For Hot Cut/Transition Detailed Methods of Procedures (DMOPs).
Action Required:
YES: / Ö / NO:
Description of Bulletin:
The purpose of this update to the original 04-01 bulletin is to include steps to identify the cables to be transitioned. This bulletin will provide all Qwest-approved installation Service Suppliers and supporting organizations with the acceptable and preferred workmanship requirements needed when performing “hot cutover” procedures on power equipment in Qwest central office locations. This includes Power Board Distribution (PBD) bays, Power Distribution Frames (PDF), Battery Distribution Fuse Board (BDFB) bays, Ringing/Tone (RT) plants, multiplexers, pairgain systems, switching, and all types of network equipment.
METHODS & PROCEDURES REQUIRED WHEN PERFORMING
“HOT CUTOVER” INSTALLATIONS
(DC POWER TRANSFER FROM EXISTING TO NEW EQUIPMENT)
Precautions required prior to performing the actual work:
1. The Installation Supplier shall not allow installers to work in a Qwest power room environment until they are familiar with power room hazards and proper work procedures. Refer to TP 77350.
2. The Installation Supplier shall comply with all Occupational Safety and Health Administration (OSHA), National Electrical Code (NEC), and local safety rules.
3. Although low AC and DC voltages are not considered “hazardous” or “high voltage”, they can still cause serious injury or death if proper precautions are not followed. A nominal value of -48 VDC qualifies as a “low” voltage.
4. The Installation Supplier shall use insulated tools and protective materials in the power room environment to prevent the occurrence of accidental electrical contact (shorting) of working equipment to ground. Qwest recommends the use of rubber floor mats and or Electra-Shield Blankets for personal protection from electrical shock while performing work on or near live equipment.
5. Safety glasses must be worn at all times during job activities.
6. The Installation Supplier must remove rings, watches, dangling keys, jewelry, etc. during all job activities in the area of power equipment and batteries.
7. The use of metal measuring tapes in Qwest power room facilities is strictly prohibited.
8. The Installation Supplier shall exercise cautious awareness when working around or near equipment with rotating components (e.g., diesel generators) to reduce the chance of loose clothing becoming entangled.
9. For personal safety, a minimum of two people shall be present during “hot cutover” installation work. Both individuals will be aware of the potential for hazards and disruption of service, both will be trained in the proper measures required to avoid accidents, and will demonstrate knowledge of the actions necessary in case of an emergency.
10. The Installation Supplier shall prepare a written detailed method of procedure (DMOP) that includes, but is not limited to, the following requirements:
a) Detailed description of what this work will accomplish.
b) Equipment being worked on and the effect on any other equipment.
c) Safety requirements for personnel, equipment, tools and (if applicable) hazardous material.
d) Identification of the personnel performing the work operation(s), their on-site work hours and the subject-matter-experts (SME’s) available on call.
e) Identification of equipment being worked on, critical circuits, alarm responsibility, fusing and location(s) of equipment “spares
f) Identification of calibrated test set(s), test descriptions, and expected test results.
g) Emergency service restoral plans
h) The (DMOP) shall be reviewed on site before actual work begins. The vendor will step through each step of the DMOP with Qwest to ensure no changes have been accommodated without the Qwest power maintenance approval and accuracy.
Identification of all cables to be transitioned:
The first step would be to verify the accuracy of the word doc (DWP) outlining the circuits to be transitioned. Next the cables need to verified (do not trust the fiber tags accuracy). The first choice would be to do a non interrupted trace of the individual cable from the Circuit Protection device (fuse/circuit breaker) to the Equipment by one of two means. The “Ring” method would be placing a non conductive ring (eg. nylon cable tie) around the cable at one end and moving it the entire length of the cable run to the Equipment termination point. This method will require all stitching to be removed from the cables. The other method would be the “hand or touch” method. This would require you keeping a constant “touching contact” with the cable as you trace the cable from one end to the other. Label the cables at the transition point as the cables are positively identified. Make sure the transition point location you choose will be adequate for you and your help to safely work the transition.
In situations where we are unable to perform an uninterruptible “ring” or “hand” trace from the circuit breaker/fuse to the equipment due to cable buildup or an accessibility issue we would want to make all connections prior to powering down the original cb/fuse to insure we did not accidentally power down any equipment we were transitioning. We still need to do a “ring or hand” trace from the equipment end or the CB/Fuse end to the transition point and mark the cables as we positively identify them to transition point. Performing the individual testing steps in the DMOP section will validate cabling accuracy and help insure Network Reliability.
Check the new cables that will be transitioned for identification accuracy before proceeding also.
Precautions required while performing the actual work:
1. The Installation Supplier shall verify that the polarity of the components to be connected is correct per manufacturer’s requirements before any power-related “hot” cutover connection is made.
2. Using a calibrated/certified Digital Volt Meter (DVM), the Installation Supplier shall verify that the voltage differential between the two connection points (at the H-tap) is less than or equal to 0.05VDC this change is due the sensitivity of some network elements. The calibration date listed on the test set’s calibration sticker should not exceed one year prior to this work procedure being performed.
3. RTN cables.
A. Using a clamp on DC Amp meter take a current reading at the equipment end, and at the original Power Source end, these two readings should be the same. Next check with a DVM to verify Voltage difference is equal to or less than .05VDC. After H-Tapping the RTN cables together take a current reading on the equipment side of the H-Tap and at the original RTN Buss and the New RTN buss. The totals of the two RTN buss readings should equal the equipment current reading. Depending on where the new plant is located and the added cable you could see a shift in the load from the original cable to the new cable. If the current readings do not equal the equipment reading stop, and check your cabling, you may have H-Tapped into the wrong RTN cable.
4. Battery cables
B. Make sure the New Power Plant Circuit Protection device is not energized. Using a clamp on DC Amp meter take a current reading at the equipment end, and at the original Power Source end, these two readings should be the same. Next check with a DVM to verify Voltage difference is equal to or less than .05VDC. Next H-Tap these Battery cables. Before energizing, check with your DVM across the Protection Device to make sure the voltage difference is equal to or less than .05VDC. You might need to tweak the new plants voltage up or down. Close the breaker or install the fuse. Take a current reading at the old cb/fuse position and at the new cb/fuse position. The total should equal the current reading from the equipment side of the H-Tap. Next increase the voltage on the new plant by .05VDC; this should cause the current to shift from the old plant to the new plant. Verify this with the amp meter. Close the H-Tap covers per 77350 7.4.5 Next de-energize the original Circuit Protection Device (fuse/cb)
C. You may now proceed with cutting the old feed cables from the H-Tap to the original power plant. Use heat shrink end caps on the stub ends of both BATT and RTN cables and secure to the cable. Be sure to protect all exposed cables and H-Taps during this process to protect the ends from possible shorting by using heat shrink end caps or insulated tape.
5. While work is being performed on energized circuits of 60VDC or greater, the protection device (fuses or breakers) shall be removed/switched open to remove the voltage potential if it is possible to do so without jeopardizing service. The Installation Supplier shall provide a method of “lock-out/tag-out” as noted in TP 77350, Par. 9.1.12.
6. On AC-related power work, all non-essential circuits (e.g., frame/aisle lighting and appliance outlets shall have their circuit breakers switched off and power removed while work is performed.
7. The Installation Supplier shall wrap live bus bars and cover live terminals/lugs near power equipment being worked on with insulation blankets per Quality Bulletin 02-01 dated January 25, 2002. http://ecom-co37.uswc.uswest.com/techpub/.
IOT required text for Detailed MOPs: Hot Cut/Transition work
Step 1:
Please distribute this Detail MOP to: (CO Manager Name)
Central Office Phone Number: XXX-XXX-XXXX
Central Office Address: XXXXXXXX
Work will be performed on mm/dd/yy from hh/mm to hh/mm
Job Type: Switch/Power/IOF/Other
Design Engineer: DE NAME, Design Engineer Phone# XXX-XXX-XXXX
PNAR#: XX-XX-XXXX
Step 2:
This detail MOP is to cover the transition of the following equipment from the original PBD to new PBD. The service provider and the assigned network technician will review all steps and agree to the work to be performed prior to starting this work.
Equip / Ckt / old PBD / PNL / CB/Fuse / new PBD / PNL / CB/Fuse / Fused @PCFD01 / LD-B / 001.1 / 101 / 0100.01 / A / 112 / 225A
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
XXXX.XX / XX-X / XXXX.XX / XX / XXX / XXXX.XX / XX / XXX / XXXA
Step 3:
All ENERGIZED POWER work will be performed and completed within the Maintenance Window. SAFETY and INSTALLATION guidelines of QWEST Publication 77350 shall be conformed to.
Step 4:
Contact the Power Alarm Center @ 1-800-713-3666 (direct number) or 800-341-8188 Opt 1 (Switch/Switch Power/Power) and have them do an office scan, note any alarms present and get on the White Board before beginning work. YES/NO/NA Exposed surfaces will be protected with Rubber Insulating mats and/or Electro-Shield Insulating blankets. Insulated tools will be used. YES/NO/NA
Step 5:
Have You Considered?
Added or removed equipment with potential to impact service or working equipment? Eye wash, battery spill kit, First Aid available? Affected Working Circuits? Equipment Compatibility? Work Area Protection? Special Tools/Insulated tools/materials? Hazardous materials handling and disposal? Location of spare fuses? Before and after tests? Emergency restoration plans? All applicable Product Change Notices have been reviewed/applied? Establishing a bridge with the NROC to monitor the office? Office records/drawings available and corrected? Documents referenced in the MOP on site? Back out procedures? Fuse alarm operation? Personal Protection Equipment? YES/NO
Step 6:
Prior to commencement of work operation(s), I have requested proper coverage. CO Operations Representative notified? Has everyone who needs to know about the impact of this work been notified? YES/NO. I am trained and qualified to do this work? YES/NO. Is the supporting documentation current and error-free to minimize all risks associated with this task? YES/NO
Step 7:
I have everything I need to quickly restore service if something goes wrong? YES/NO
Step-by-step "backout" procedures? YES/NO Cables tagged for ease or reconnection? YES/NO Spare fuses of the proper amperage for the equipment relevant to this DMOP? YES/NO The proper material (e.g., lugs, screws/bolts/nuts, connectors, cable/wire, etc.). YES/NO I am using the right tools to perform this work? YES/NO
Step 8:
In the event of a service interruption, I will complete the RG47-0013 and notify: Qwest Network Event Management Center: 800-879-1200, Qwest CO Operations and my immediate Supervisor. YES/NO If the event caused an interruption of service to an Electronic Switch (Lucent, Nortel or Ericsson) DO NOT RESTORE SERVICE, but contact NEMC at (800-879-1200) for assistance with the Power Restoration Process. Working equipment that has been deactivated/powered down/removed from service in order to complete these procedures will be placed back in service. YES/NO
Step 9:
I have identified the locations of essential and/or government circuits such as 911, Federal Aviation Administration (FAA) and System Signaling 7 (SS7). YES/NO/NA I have previously submitted the PNAR and notified NROC, so that they could initiate release requests to ensure proper authorization (Required for upgrades/retrofits that do not use the CIA process. The Service Supplier must initiate the PNAR at least 10 days prior to performing the upgrade/retrofit so that the NROC can obtain the releases). YES/NO/NA
Step 10:
I have ensured that these circuits have been properly identified, labeled, and that the NROC has contacted the appropriate representatives to obtain the proper work release authorization(s). YES/NO/NA Note: Use the Search Tool http://pas16.uswc.uswest.com/omni/docview.html IF YOU ANSWERED "NO" TO ANY OF THE ABOVE -- CALL YOUR SUPERVISOR PRIOR TO JOB START