Office of Enforcement and Compliance Assurance
Office of Enforcement and Compliance Assurance
Integrated Compliance Information System
ICIS Batch
Single Event Violation – Technical Specification
Version 1.2
Revised Final
TBD
- 1
Office of Enforcement and Compliance Assurance
Document Change History
Version Number / Date / Description0.1 / March 14, 2011 / Initial Draft Release.
1.0 / May 13, 2011 / •Incorporated comments provided by EPA
•Updated text for RNC to Reportable Non Compliance when referring to XML tag names.
1.1 / March 16, 2012 / Updated the document based on the redesign agreed upon by Booz Allen and EPA:
•Rewrote the business rules to be shared among DMR Violations, Schedule Event Violations, and Single Event Violations. They will now share the same error codes when applicable. Updated the business rules and error messages. Added new rules, edited rules, and removed rules. The old rules can be found in Version 1.0 of the technical specification.
•Removed the sub-flow Process Generate RNC Information.
•Revised the example scenarios as needed to reflect the redesign changes.
Other document updates:
•Clarified the text that explains default values in Section 2.1.2. Separated web-only fields from tags that are submitted in batch. Clarified that default values apply to New, Change, and Replace transactions. Clarified that if a field is blanked out on a Change or Replace like Change transaction, it will be defaulted in batch.
•Added new business rule SEV045 to reject if multiple Single Event Violations exist for the key data submitted. New steps were added to the flow diagrams for Mass Delete, Change, and Replace.
•Updated the Background Processing trigger for Change and Replace. Removed the condition where RNC Resolution Code = 1.
•Added a new business rule SEV095 that says that the Single Event Violation date must be less than or equal to the current date. Applies to N and R like N.
•Updated the text in the business rule column for VIO090. Changed REF_RNC_DETECTION_TABLE to REF_RNC_DETECTION table.
•Removed the note from SEV050 that says that "This business rule will be implemented for batch processing, but will not be included in the web until O&M CR 540 is implemented".
•Removed business rule SEV070 that said that the NPDES ID entered cannot have a Permit Status of Pending. This rule is already covered by business rules SEV090 and SEV095.
•Added a new business rule, SEV085, that applies to New and Replace like New and that says the original Permit Effective Date must exist if the Permit Type is not Unpermitted Facility.
•Reworded the business rule column and error message for SEV090. Added "if that Effective Date exists" to account for Unpermitted Facilities and reworded the rule and error message to clarify its meaning.
•Updated the business rule column of VIO100. Made it clearer that this rule only applies to Effluent and Single Event Violations.
•Removed the BGP trigger for EA RNC Processing that says to only call the process when the Violation is Non-Compliant and linked to one or more Final Orders. EA RNC Processing is now called for every saved Violation.
•Updated the supporting table for the flow diagrams for New and Replace like New. Specified which version of the permit newly added Single Event Violations are attached to.
•Updated the error message for SEV080 to be consistent with other ref table check error messages.
•Updated the Replace like New flow diagram to more closely resemble the New flow diagram and the application code. Moved the privileges check to be after the check to see if the NPDES ID exists.
1.2 / TBD / EP-384: removed Single Event Violation Start Date tag:
•Updated sample scenarios to remove Single Event Violation Start Date tag
•Removed business rule SEV110
•Updated business rules SEV030, SEV040, SEV045, SEV090, SEV095, SEV120
•Updated Data Element Mapping to remove Single Event Start Date tag and updated screen name for Single Event Date tag
1
ICIS Batch – Single Event Violation
Office of Enforcement and Compliance Assurance
Table of Contents
1. Introduction
1.1 Purpose
1.2 Assumptions and Constraints
1.3 Document Overview
2. Validation and Processing
2.1 General Batch Processing Rules
2.1.1 Asterisks
2.1.2 Default Values
2.2 Mass Delete (X) Single Event Violation Processing
2.2.1 Mass Delete Single Event Violation Processing Flow
2.2.2 Mass Delete Single Event Violation Sample Scenarios
2.3 New (N) Single Event Violation Processing
2.3.1 New Single Event Violation Processing Flow
2.3.2 New Single Event Violation Sample Scenarios
2.4 Change (C) Single Event Violation Processing
2.4.1 Change Single Event Violation Processing Flow
2.4.2 Change Single Event Violation Sample Scenarios
2.5 Replace (R) Single Event Violation Processing
2.5.1 Replace Single Event Violation Processing Flow
2.5.2 Replace Single Event Violation Sample Scenarios
2.6 Business Rules
3. Data Element Mapping
4. XML Schema
Appendix A: Acronyms
Appendix B: XML Submission Sample
1
ICIS Batch – Single Event Violation
Office of Enforcement and Compliance Assurance
List of Tables
Table 21: Mass Delete Single Event Violation Processing
Table 22: Mass Delete Single Event Violation Example 1
Table 23: Mass Delete Single Event Violation Example 2
Table 24: New Single Event Violation Processing
Table 25: New Single Event Violation Example 1
Table 26: New Single Event Violation Example 2
Table 27: New Single Event Violation Example 3
Table 28: Change Single Event Violation Processing
Table 29: Change Single Event Violation Example 1
Table 210: Change Single Event Violation Example 2
Table 211: Change Single Event Violation Example 3
Table 212: Replace Single Event Violation Processing
Table 213: Replace Single Event Violation Example 1
Table 214: Replace Single Event Violation Example 2
Table 215: Replace Single Event Violation Example 3
Table 216: Replace Single Event Violation Example 4
Table 217: Single Event Violation Business Rules
Table 31: Batch Single Event Violation Data Element Mapping
Table A1: Acronym List
List of Figures
Figure 21: Mass Delete Single Event Violation Processing Flow
Figure 22: New Single Event Violation Processing Flow
Figure 23: Change Single Event Violation Processing Flow
Figure 24: Replace Single Event Violation Processing Flow
1
ICIS Batch – Single Event Violation
Office of Enforcement and Compliance Assurance
1. Introduction
Many states already have their own tracking systems for National Pollutant Discharge Elimination System (NPDES) data. When those states are migrated to the Integrated Compliance Information System (ICIS), having them enter these data in ICIS through the Web interface would require them to perform duplicative data entry. Instead, users from these states may enter data in ICIS through batch submissions by which they extract data from their state system and submit it to ICIS in Extensible Markup Language (XML) files. The type of data a state submits through batch depends on the type of data tracked in their state system.
Current users of ICIS include Headquarters, Regions, direct-user states, hybrid states using Batch Discharge Monitoring Report (DMR) functionality, and full batch states. Batch modules are being added in groups, and the Single Event Violation module is part of Full Batch Release 3. Single Event Violations are violations entered manually by users.
The focus of this technical specification is the submission of Single Event Violation data to ICIS through batch XML transactions. Data for other areas of the ICIS system (e.g., Formal Enforcement Actions, Compliance Schedules, and Enforcement Action Milestone) will be addressed in separate technical specifications.
The general process for states to submit batch data was defined for batch DMR processing and remains unchanged. States do not submit batch data directly to ICIS. Instead, batch files are submitted to the Environmental Protection Agency’s (EPA’s) Central Data Exchange (CDX) which then passes the files to ICIS. To submit data to CDX, the state must have a CDX User ID and password. This ID and password are specific to CDX and are completely unrelated to ICIS. An ICIS User ID must also be provided in the ID tag in the header of each XML file so that when ICIS receives the batch file(s), it can determine if the transactions in the file can be performed by the user submitting the batch. After receiving a batch from the state, CDX performs several important functions. They perform a virus scan to ensure that the state files are free of viruses and then assign a unique Transaction ID to the batch. This Transaction ID maps directly to the Batch ID that ICIS uses internally to manage processing. ICIS uses this Transaction ID to communicate information about the batch to CDX and the state. CDX then archives the batch and validates the XML files based on the rules in the XML schema. If problems are detected, CDX notifies the state so that the problems can be corrected. Upon completion of these tasks, CDX sends the error-free batches to ICIS.
For purposes of this document:
•“CDX User ID” refers to the ID the user must have to log in to CDX.
•“ICIS User ID” refers to the state user’s ID in the ICIS system.
•“Transaction ID” refers to the identifier CDX provides for each batch.
•“Batch ID” in all communications with users (e.g., audit reports, batch processing confirmation report) refers to the identifier CDX provides for each batch (i.e., the Transaction ID).
•“Batch ID” in the ICIS Batch Operational Database (DB) refers to the batch identifier assigned by ICIS to make processing more efficient.
•“Payload” in a batch refers to all of the XML transactions for a submission type.
A batch may contain many XML files, and within each XML file there can be up to one Payload for each Submission Type (Single Event Violation). Each Payload may contain one or many XML transactions, each of which contains the Single Event Violation data and a specific transaction type that identifies how ICIS should process the data. For Single Event Violations, there are four valid XML transaction types: Mass Delete, New, Change, and Replace. The details of the Single Event Violation transaction types are described in Section 2: Validation and Processing. After receiving a batch from CDX, ICIS parses it and saves each Single Event Violation XML transaction to the database so that the individual Single Event Violation XML transactions can be ordered and processed. After processing is complete for all files in a batch, ICIS sends a response notification to CDX, which then notifies the state, regional, or headquarters user when processing is complete.
1.1 Purpose
The purpose of this document is to provide a comprehensive overview of the submission of Single Event Violation data through batch XML transactions using text descriptions, tables, and figures.
A major section of this Single Event Violation Technical Specification, Section 2: Validation and Processing, details the four valid XML transaction types for Single Event Violation (Mass Delete, New, Change, and Replace). Provided with these transactions are the business rules that govern batch Single Event Violation transactions, as well as the accompanying error/warning messages, serving to notify users of the data in error and provide them with the information necessary to correct the problems.
1.2 Assumptions and Constraints
The following assumptions and constraints apply to the ICIS Batch Single Event Violation Technical Specification:
•ICIS will process batches within a state in the order they are received from CDX. CDX will not apply a timestamp to each batch that is submitted by the state. In addition, CDX cannot guarantee that batches will be sent to ICIS in the same order that CDX received them from the state. As a result, ICIS cannot guarantee that batches will be processed in the order in which the state submitted them.
•Users will submit batch files to CDX in the correct chronological order. A procedure will be put in place by each state to ensure that a state sends only one batch at a time, and does not send a new batch until they have received confirmation that the previous batch has been processed.
•CDX will perform a schema validation on every batch. ICIS will not perform another schema validation. If schema errors exist that are not caught by the CDX validation, unexpected results will occur.
•The business rules for Single Event Violations entered via batch should be the same as Single Event Violations entered via the web application. Any differences will be noted in the “Where Enforced (Web)” Column of the Business Rules table in Section 2: Validation and Processing.
•Design decisions will be made to minimize software changes that will be needed to incorporate the batch entry of other data families. EPA will be consulted, as appropriate, as these decisions are made.
•ICIS will not save a transaction if there are any errors within the transaction, though transactions will be saved if only warnings/informational messages exist. The rules for accepting and rejecting transactions are described in detail in Section 2: Validation and Processing.
1.3 Document Overview
The following sections comprise the remainder of this technical specification:
•Section 2: Validation and Processing – This section describes the processing of Mass Delete, New, Change, and Replace Single Event Violation XML transactions, and the business rules that apply to each transaction type.
•Section 3: Data Element Mapping – This section provides a mapping between the Permit Compliance System (PCS) Acronym, XML Tag Name, ICIS Screen Name, and ICIS Database Name for Formal Enforcement Action data elements.
•Section 4: XML Schema – This section provides a list of the XML schemas related to Single Event Violation.
•Appendix A: Acronyms – This section provides a list of all acronyms used in the document.
2. Validation and Processing
After receiving a batch from CDX, ICIS parses the batch Single Event Violation XML transactions, saves each to the database, and groups them by transaction type. The valid transaction types for Single Event Violation are Mass Delete, New, Change, and Replace. ICIS must process these groups in the proper order to achieve the desired results. The ICIS Batch Design Document Appendix D: ICIS Batch Submission Types and Processing Order details the processing order for all ICIS batch submissions. This section describes specific fields that require special processing as well as the detailed processing of the four Single Event Violation transaction types.
Key values are used throughout ICIS Batch to identify data in the ICIS database. The ICIS Batch Schema requires that key values be entered for all transactions. The key values for Single Event Violation transactions are:
•Permit Identifier
•Single Event Violation Code
•Single Event Violation Date
The following sub-sections describe general processing rules related to asterisks and the detailed processing of Formal Enforcement Action XML transactions.
2.1 General Batch Processing Rules
It is important to understand the following general batch processing rules for ICIS:
2.1.1 Asterisks
Users must have the ability to remove data from fields through batch transactions. This is accomplished through the use of asterisks (*). The asterisk is not stored in ICIS, but instead is used to signal the system that the field should be blanked out. Asterisks can also be submitted in Replace transactions. When that happens, they are interpreted by the system as blanks. It is not necessary for the user to submit asterisks in Replace transactions because leaving the tag out of the XML transaction achieves the same result (i.e., the field is left blank), but ICIS has been designed to process asterisks in those cases.
2.1.2 Default Values
Default values can include both web-only fields and web/batch fields. They apply to New, Change, and Replace batch transactions. Default values are always set for New and Replace like New transactions, and are set for Change and Replace like Change transactions for tags that are blanked out.
The following is a list of web-only default values for Single Event Violation transactions:
•Single Event Agency Type: defaults to U.S. EPA for Regional and HQ users and defaults to State for State users.
2.2 Mass Delete (X) Single Event Violation Processing
The Mass Delete Single Event Violation XML transaction allows the user to remove a Single Event Violation from ICIS. When a Single Event Violation is removed from ICIS, links to other records are deleted, but the actual records themselves are not deleted.