Adding a new Probe location to the CEA system
This document is a general guide to add a new location/interface type to the system. It is highly recommended to use an existing data source / location (of the same interface type) as a reference.
1.- Create a new Data Source
- Go to Data Collation & Loading -> Data Sources
- Click on the “+Add” button
In the General tab:
- Provide an Id: Unique identifier (no spaces)
- Provide a Name: User friendly name
- Specify a Type: Normally this will depend on the manufacturer of the probe
- Time Zone: Time zone used to encode data in the file
- Check “Active”
In the Alert tab:
-Check “Alert on Delay”
-Provide a Warning Threshold: Time when a Data Source will be considered late
-Provide an Error Threshold: Time when a Data Source will be considered missing.
-Click the “Save” button.
2.- Create a new Location
- Go to Data Collation & Loading -> Collectors -> Locations
- Click on the “+Add” button
In the General tab:
- Provide an Id: Unique identifier (no spaces)
- Provide a Name: User friendly name (normally landing zone + interface type)
- Select the data source created in step 1.
- Specify a Type: Indicate the type of collector that will be used for this location.
- Specify an ETL: Indicate the type of ETL that will process this location.
- Check “Active”
- Time Zone: Time zone used to encode data in the file
- Specify Landing Zone: The name of the Landing Zone (location)
- Indicate Data Type: Normally the interface type.
In the Alert tab:
-Check “Alert on Delay”
-Provide a Warning Threshold: Time when a Data Source will be considered late
-Provide an Error Threshold: Time when a Data Source will be considered missing.
In the Metadata tab
-Indicate the Host Name: Normally Location IP address.
In the Parameters tab
Source section
-Indicate the URL: Normally this will be a sftp address that indicates the landing zone and folder where the files to collect are located.
-Indicate User name: User Name to use to log-in into the landing zone
-Indicate Password: Password to use to log-in into the landing zone.
-Indicate Files newer than: Normally set it to 72
-Indicate Include file pattern: This normally will be part of the file name that is unique for all files of this type.
-Indicate Exclude file pattern: Normally will be *InProgress*
Destination section
-Indicate URL: This will be the local location where the file will be copy. Normally the location is unique for the Type of files collected and its location.
-Indicate File Pattern: Normally it will be as follow: %SOURCE_ID%.JDSU_IUPS.%FILE_TIME_YYYYMMDDHHMMSS%.5.%TOKEN(.,4)%.csv.gz
-Indicate Data Type: The type of interface.
Network section
-Indicate Connection Timeout: Normally 30000
-Indicate Read Timeout: Normally 120000
-Indicate Write Timeout: Normally 120000
Note: For most Locations, use the default value for any other parameters. Use an existing data source / location (of the same interface type) as a reference.