OPERA note

December 13th, 2002

Cabling and installation procedure:

baseline proposal

D.Autiero, J.Marteau

This note describes a proposal for the cabling strategy, electronics and DAQ installation and control room position. This strategy was discussed in a dedicated meeting at CERN (November, 28th, 2002) and updated by the project leaders boards.

1/ General overview:

The main conclusions are the following:

  1. The cables of each sub-detector (Target Tracker, Precision Tracker and RPC Tracker) and the power supply cables of the magnets will be collected on the top of the detector and connected there to their dedicated boards in crates or Ethernet repeater or supply unit. These cables going from the sub-detectors to the top of the experiment will be referred to as vertical (V) cables. To avoid interferences with the cranes operations, the racks will be limited in height (max. 1.7m). These reduced racks should be able to host 4 standard crates at most.
  2. On the top of the detector all the cables will be routed along 2 side cable trays (around 0.5m wide) to a control room (or electronic room). These cables will be referred to as horizontal (H) cables.
  3. In order to preserve access to the top PMTs of the vertical modules, it is required that the electronics above the Target Tracker is placed along the sides of the detector. In order to access to the electronics and to the cable trays, it is foreseen to extend the platform on top of the detector by 1m on each side. The cable trays will be fixed to the platform starting from that extremity. It that sketch, it is possible to walk in the space at the back of the crates.
  4. The control room will be placed at the extremity of the platform, close to the entrance of hall C. Its final position will depend on the architecture for the detector main structure. This control room will host Ethernet switches, PCs for the event building and DAQ management, PCs and racks for slow control operations. A more comfortable terminal room will be placed at the extremity of hall B.

A general sketch for the electronics, cable trays and control room positioning is displayed on Fig.1 (the figure is not to scale). The constraints of access to the PMTs do not affect the spectrometer part and therefore the space on top of the magnets (around 5m x 10m) is fully available for the electronics and power supply racks. It is foreseen to separate the racks from PT, RPC Tracker and magnets. The cables from the central racks will be collected in transverse cable trays to the sub-detectors in order to minimize possible interferences.

Figure 1: top and lateral views of the OPERA detector showing the position of the electronics, the cable trays and the control room.

Figure 2: front view of the OPERA detector. A platform is placed on the top of the detector with the electronics racks (represented in orange). The cables are routed from these racks to the control room in two horizontal trays (in red) at the extremities of the platform.

2/ Description of each sub-detector acquisition chain:

2.1/ Target Tracker

Per super-module the Target Tracker consists of 31 scintillator planes divided into 8 modules each containing 64 strips readout at both ends by MaPMTs. Each MaPMT is readout by an analogic and a digital boards enclosed inside the module end-cap. The analogic board hosts the two 32 channels F/E chips and is connected via two short flat cables to the digital board. This one includes the ADC(s), the sequencer (FPGA), the Ethernet controller, the clock receiver unit (EPLD and transceivers), the high voltage module for the MaPMT and a LED pulser circuit for the calibration.

This electronics is only powered by low differential voltage (max. 7.5V,min. 6V). The total power consumption per MaPMT (or Ethernet node in this case) is estimated to be less than 20W (which is the conservative limit assumed in the following).

In the baseline configuration, a total of 3 wires are mandatory for each node:

  • the low voltage supply
  • the Ethernet cable for the signal (control, monitoring, acquisition)
  • the clock cable

The clock cable will be a standard RJ45 Ethernet cable. It is daisy chained from one module to the following and passes along the protected path of the end-caps. The M-LVDS bus requires to connect a maximum of 8 modules (half a plane: 4 vertical+4 horizontal modules). A total of 2 cables are then routed vertically from each TT plane to the top of the detector and connected to a master clock board placed in a crate. The 2 cables are coming up from the same side and can therefore be coupled before being connected.

The Ethernet cable for the signal is connected on a RJ45 connector on the digital board and feeds the input of a hub (or a switch) placed at the vertical of the TT plane on the top of the detector. As this is a point-to-point connection, the cables from the bottom modules should go along the upstream end-caps without passing above the electronics (either inside the U ending the end-cap or along its external side). The same holds for the power supply cables. We will study a solution where the repeater on the top of the detector is replaced by repeaters integrated on each digital board. In that sketch a single Ethernet cable would be used for the signal. The figure of merit of both solutions has to be evaluated.

The collection of the cables is performed by “half planes” containing: 2 bottom vertical modules, 4 horizontal modules, 2 top vertical modules. As the cables are routed to the top, the maximal number of cables routed inside the most upstream horizontal module is 13 (6 for the signal + 6 for the power supply + 1 for the clock). In order to minimize the total number of cables to be routed along or close to the electronics, the four cables (2 for the signal + 2 for the power supply) of the bottom modules will pass along the side of the end-caps close to the scintillator. The global sketch for this cable layout is displayed on Fig.2.

The powering of each module is under study. The baseline proposal is to have dedicated supply crates grouped in 5 racks (per side) on the top of the detector (see Fig.1). Two options are investigated based either on supply units with 1 output per module or on most powerful supply units with one output per group of modules and local splitter + regulator to each module.

Table 1: detailed numbers for the TT. Numbers are given for one super-module.

Figure 3: sketch of the cables going along the TT to the top of the detector. The yellow daisy chained cable is for the clock distribution. The red and blue cables represent the power supply & Ethernet signal cables.

Summary for the TT:

The total power consumption for the TT amounts to ~32 kW for the 2 super-modules.

The total number of horizontal cables routed to the control room is (31+5+1) x 2 x 2 ~150.

The different options to be fixed concern the choice of power supply units, the choice of the hubs (or switches) at the top of the detector and/or the replacement of these repeaters by local repeaters at the controller board level.

2.2/ Precision Tracker

The electronics for the PT, the RPC Tracker and the magnet will be placed at the top of each magnet. The electronics of the PT looks compatible with a setup which is central instead of left/right symmetric. The compatibility of this design should be checked w.r.t. the cable length, possible interferences…

In the present design the cables will go from the central racks to the detectors in a dedicated transverse cable tray. The front-end electronics is divided into 3 main items: pre-amplifiers, discriminators, TDCs. In the baseline architecture, the pre-amplifiers and discriminators are placed at the chamber level. Only the TDCs and the controller boards require to be placed in racks at the top of the detector.

A PT plane has 192 tubes and a station is made of 4 planes. A spectrometer comprises 3 doublets of stations. A station requires 8 HV cables. These cables are going inside the magnet but only on the edge of the top. They can therefore be installed at any moment of the installation. The HV will be delivered by the CAEN SY2527 system.

The readout and the supply cables for the front-end could be installed before the modules installation in a cable tray going inside the magnet and passing through it ~every meter to be connected. 64 twisted pair cables (40 lines each) are required for 1 station. They include the power supply lines. On top of this, each PT station needs the t0 signal coming from 2 XPC planes or 2 nearby RPC planes. The collection and pre-processing of the data will be done at the level of the controller board (with the F/E interface, the sequencer, the FIFO, the processor with Ethernet interface, the clock receiver). In the baseline design, we foresee 1 controller board for 4 TDC boards. These controller boards will be 9U boards, hosted in the same crates as the TDC boards (e.g. 16 TDC boards + 4 controller boards per crate). A total of 12 controller boards is required. Even if the trigger is given by the OR of 2 RPC planes, if we transmit only the events where a signal is available on the tubes, the data rate will be low. We could therefore connect the Ethernet output of each controller board on a hub (or a switch). A maximum number of 2 Ethernet repeaters is required per SM. The power of the controller boards is taken from the VME crate itself.

The slow control strategy, although not completely defined includes:

  • temperature monitoring (1 serial bus per station, connecting sensors in the x & y direction); the collection of the data is done by a PC running Linux, to be placed near the racks or in the control room.
  • HV monitoring through an Ethernet connection from the CAEN system to a PC
  • TDC crates power supply monitoring (CAN bus)
  • Power supply for the pre-amp & discriminators monitoring (CAN bus)

The slow control implies therefore 6 additional (V) cables for temperature monitoring and 8 (H) cables for the power supplies monitoring.

All the numbers for one SM are recalled in Table 2.

Table 2: detailed numbers for the PT. Numbers are given for one super-module.

Summary for the PT:

The total power consumption for the TT amounts to ~30 kW for the 2 super-modules.

The total number of horizontal cables routed to the control room is (11) x 2 = 22.

The slow control interface to the central DAQ remains to be defined.

2.3/ RPC Tracker

The RPC tracker readout and power supply require 6 racks per super-module occupying the corners of the platform above the spectrometer (see Fig.1). 4 racks are devoted to the readout for:

  • Discriminator boards,
  • Trigger boards,
  • Controller boards,
  • Master clock boards.

9 front-end boards (discriminator boards) are mandatory to readout one plane (562 channels). Each front-end board has 64 channels and the trigger is defined by the fast OR of 16 channels. The front-end boards are in VME standard and are connected in 10 crates. From the racks down to one RPC plane there will be 9 x 4 flat cables, all collected from the same side of the detector. In order to symmetrize the distribution we will have 2 racks on each side of the detector.

The trigger will be send to the controller board (one per RPC plane) for time stamping and to a dedicated trigger board (two per RPC tracker) that will perform a majority trigger. This hardware trigger is a conservative approach and could be switched off if the data rate from all the chambers is manageable. The clock is also distributed to the controller boards. As they are closed to each other, we can connect 11controller boards together (the M-LVDS bus can be extended to 30 connection nodes). Two crates (controller crates) with 11 controller boards + 1 trigger board + 1 clock board are placed into the previous racks.

2 racks (1 per side) are used for the RPC HV supply. Each RPC plane is supplied by 7 HV cables. The system used for the HV distribution is still to be defined.

All the numbers for the RPC tracker in one super-module are displayed in Table 3.

Table 3: numbers of channels and cables for RPC tracker (for one super-module).

Summary for the RPC Tracker:

The total power consumption for the TT amounts to ~20 kW for the 2 super-modules.

The total number of horizontal cables routed to the control room is (36 x 2) = 72.

The slow control interface to the central DAQ remains to be defined.

2.4/ Magnets

The magnet power supplies will be placed in 2 racks on one side of the top platform. They will be connected with two big HV cables and at least one slow control cable. Details have to be defined.

2.5/ B.M.S.

The BMS has 4 connections to the control room which pass in special trays above the top rail of the portico. This cabling chain is quite independent from the rest of the sub-detectors.

3/ Control room

The counting room will host the event building + slow control system. The Ethernet cables for the signal readout and control (2 times (62 + 2 + 22) = 172 cables) are gathered in 4 switches (48 inputs, Gigabit output) which outputs are connected to the event building PCs. The slow control systems (not yet completely defined) are just estimated in terms of equivalent racks for the different sub-detectors. The equipment required for the control room is summarized below:

  • 4 switches in half a rack
  • PCs:
  • 3 for the global event building + manager
  • 2 for RPC & magnet control
  • 3 for PT control
  • 2 for BMS control
  • slow control racks:
  • 2 for PT
  • ½ rack for RPC
  • ½ rack for magnet
  • ½ rack for local RPC DAQ (to be used mainly during the installation phase)
  • 1 rack for the BMS

The approximate surface needed for the control room(~15 m2) is compatible with a setup where it is put on the top of detector, close to the readout and control electronics.