Request for Project Closeout Certification Form
1. Project Governance[1] /Project Name
Short Name(s) / OSE/ISC Water Rights Business Process Management System; or,
BPMS Project
Date / November 6, 2015
Lead Agency / Office of the State Engineer/ Interstate Stream Commission (OSE/ISC)
Other Agencies / None
Executive Sponsor
/ Charles Kinney, CIO & Information Technology Services Bureau ChiefJohn Romero, Director, Water Rights
Agency Head / Tom Blaine, PE, State Engineer
Agency CIO/IT Lead / Charles Kinney, CIO & Information Technology Services Bureau Chief
Project Manager / Charles Kinney, CIO & Information Technology Services Bureau Chief
2. Project Abstract[2]
A. Brief project description and purpose /
For OSE/ISC, the overall BPMS project objective was to redesign and automate water rights administration business processes with BPMS software and to replace an existing system known as WATERS. BPMS software platforms are designed to model, design, and implement software applications to manage business processes from start to finish. The project was intended to proceed in phases, described in Table 2B.
For DoIT, the project was certified to enable a significant trial of a BPMS platform for business process automation.
Although project phases 5-8 (Table 2B, below) were not completed, successful implementation of the OSE/ISC Well Driller application (phase 9) revealed strengths and weaknesses of the BPMS platform and will usefully guide future BPMS projects (lesson learned 4). Successful design, development, and deployment of a new Water Rights Tracking application (phase 10) drew directly on analyses that were completed during project phases 1-4.
2. Project Abstract
B. Project Phases and Outcomes /
Phase / Deliverables / Outcome & Comment /
1. / Planning & Envisioning / Current State Analysis
Target-State Analysis
Architectural Roadmap / Completed.
These documents will provide useful information for future systems development.
2. / Technology Selection / Prepare RFI
Evaluate RFI Responses
Conduct Technology Proof of Concept
Revise Business Case
Acquire Funding / Completed.
3. / Business Project Mgt. Education / a. Requirement Specification Training
b. Business Process Definition
c. Use Case Specification
d. System Analysis Element Specifications
e. Business Process Reengineering
f. Software Development
g. Solution Architecture Modeling / Tasks a-c: Completed.
Tasks d-g: Partially completed.
These documents will provide useful information for future systems development.
Agency Water Rights and IT staff received business process analysis and reengineering training, which will be useful for future development work.
The agency recognized need for and hired business analysts in two bureaus.
4. / Architecture Proof of Concept / Conduct Proof of Concept
Select Solution Delivery Approach
Select Solution Operations Approach
Negotiate and Execute Vendor Agreements / Completed: Appian selected as BPMS vendor.
Appian is a solid BPMS platform; it is available to OSE/ISC thru June 2017 for future testing and development.
5. / Domestic Water Rights Process (Release 1) / Release 1: Build, Test, Train, Deploy
Release 2: Business Process Model & Use Cases / Phase 5, 6, 7: not completed.
Please see lessons learned.
6. / Non-Domestic Water Rights Process (Release 2) / Release 2: Build, Test, Train, Deploy
Release 3: Business Process Model & Use Cases
7. / Non-Domestic Water Rights Process (Release 3) / Release 3: Build, Test, Train, Deploy
Release 4: Business Process Model & Use Cases
8. / Non-Domestic Water Rights Process (Release 4) / Release 4: Build, Test, Train, Deploy
9. / Well Driller Application (WDA) / Not expressly within original project scope, but added when development and deployment of domestic and non-domestic water rights process application proved too difficult.
Design, develop, deploy WDA to replace manual processes. WDA tracks licensing of NM well drillers. / Completed: WDA is now in use.
Please see lessons learned.
10. “ / Water Rights Application tracking / Not expressly within original project scope, but added after successful deployment of WDA.
This project transitioned into a successful project to replace the legacy, spreadsheet-based “Reynolds Report” with a database driven, web-enabled application. / Initial deployment: October 19, 2015.
Analysis and specifications performed for domestic water rights processing (phases 1 and 3, above) informed this project.
3a. Scope Verification /
Requirements Review
/ Yes / No /Explanation/Notes
/Were the project objectives (expected outcomes) accomplished? / X / See Charter, section 2.2, for overall business objectives:
1. improve business agility;
2. improve staff productivity;
3. improve business process transparency and performance management;
4. reduce system maintenance and development costs;
5. improve constituent satisfaction.
Although the project yield solid “lessons learned,” these objectives were not met.
Were all Deliverables submitted and accepted? / X / See Charter, section 3.1, for list of deliverable documents, which range from the initial Charter, to monthly progress reports, to closeout report. Project documents prior to this document were delivered.
Did the IV&V vendor verify that all deliverables met requirements? / X / An initial IV&V report was delivered, Jan. 2, 2013.
A planned, post-release IV&V report was not produced.
Subsequently, the IV&V contract was not renewed because of cabinet secretary and CIO leadership transitions and project reassessment.
Have all contracts been closed? / X
Have all final payments been made (i.e., invoices paid) / X
Have all project documents/records been appropriately stored for future reference? / X
Has knowledge transfer been completed? / In progress.
Note to Table 3B (below). Rows 1-26 in Table 3B are copied from the Change Request for Certification and Release of Funds Form, April 15, 2013, “Major Deliverable Schedule and Performance Measures.”
3b. Major Project Deliverables and Performance Measures MetNo. / Major Project Deliverable & Performance Measure
(Explain if deliverable or measure was not met)
/ PO No. / Budget /Date
/Project Phase
1. / Establish Cloud Subscription Agreement / n/a / -0- / 06/01/12 / Contracting2. / Certify Project / n/a / -0- / 04/24/12 / Planning
3. / Purchase User & External Logon Licenses (Year 1) / 9736 / $428,000 / 06/01/12 / Contracting
4. / Conduct Training (5 Day, On-Site) / 9907 / $32,295 / 07/13/12 / Training
5. / Conduct Release 1 Implementation Planning / 10009 / $38,000 / 09/25/12 / Planning
6. / IV&V Initial Project Assessment / 10309 / $8,000 / 12/14/12 / Planning
7. / Conduct RFP for Release 1 Implementation Services / n/a / $0 / 01/29/13 / Contracting
8. / WATERS Database Design / n/a / $0 / 03/15/13 / Design
9. / Release 1 Kickoff & Orientation / n/a / $0 / 02/15/13 / Design
10. / Release 1A Requirements & Design Specifications / $41,106 / 02/28/13 / Design
11. / Set Up Cloud Infrastructure / $40,600 / 04/21/13 / Development
12. / Release 1A Build & Unit Test / $87,475 / 04/03/13 / Development
13. / Release 1A UAT, Training & Deployment / $22,885 / 05/14/13 / Deployment
14. / Release 1B Requirements & Design Specifications / $44,735 / 04/15/13 / Design
15. / Release 1B Build & Unit Test / $89,385 / 07/05/13 / Deployment
16. / Purchase User and External Logon Licenses (Year 2) / 11071 / $142,000 / 06/30/13 / Contracting
17. / Release 1B UAT, Training & Deployment / $22,899 / 08/09/13 / Deployment
18. / Release 1C Requirements & Design Specifications / $30,485 / 05/24/13 / Design
19. / Release 1C Build & Unit Test / $62,700 / 07/19/13 / Deployment
20. / Release 1C UAT, Training & Deployment / $17,674 / 08/23/13 / Deployment
21. / Release 1D Requirements & Design Specifications / $30,499 / 07/26/13 / Design
22. / Release 1D Build & Test / $71,250 / 08/30/13 / Deployment
23. / Release 1D UAT, Training & Deployment / $49,485 / 09/27/13 / Deployment
24. / Conduct Advanced Appian BPMS Training / $25,000 / 11/29/12 / Training
25. / IV&V Post Release 1 Assessment / n/a / $8,000 / 12/31/13 / Evaluation
26. / Conduct Release 2 Implementation Planning / $28,000 / 11/29/12 / Planning
TOTAL / $1,320,473
Note to Table 4A (below). Rows 1-6 in Table 4A are copied from the Change Request for Certification and Release of Funds Form, April 15, 2013, “Certification History.”
4a. Certification HistoryNo. / Date / Amount / Funding Source(s) – Funding Total Amount
(Cite specific laws, grants, etc. and total amount)
1. / April 2012 / $246,000 / Laws of 2011, Chapter 179, Section 4
2. / April 2012 / $220,000 / Laws of 2012, Chapter 19, Section 7
3. / April 2012 / $222,250 / Laws of 2012, Chapter 19, Section 4
4. / April 2013 / $400,000 / Laws of 2013, Chapter 227, Section 7, Item 15
5. / April 2013 / $70,000 / Laws of 2012, Chapter 19, Section 4
6. / April 2013 / $165,000 / Laws of 2013, Chapter 227, Section 4
TOTAL / $1,323,250
5. Schedule and Budget*
Planned Start Date / Feb. 2012 / Actual Start Date / June 2012
Planned End Date / June 2014, Release 1
TBD, Release 2 / Actual End Date / October 19, 2015
(WRRR Deployment)
Planned Cost: (Budget) / $ 1,320,473 / Actual Cost: (Total) / $1,258,602
§ Professional Services / $892,473 / § Professional Services / $811,477
§ Hardware / -0-
(from agency budget) / § Hardware / -0-
§ Software / $428,000 / § Software / $447,125
§ Network / -0-
(from agency budget) / § Network / -0-
§ Other / -0- / § Other / -0-
* Scheduled dates based on project budget, dated October 2012.
6a. Independent Verification & Validation(Include summary of the last IVV Report or Risk Management Summary Report.)
An initial IV&V report was delivered, Jan. 2, 2013. A planned, post-release IV&V report was neither requested nor produced. The IV&V contract was not renewed subsequent to cabinet secretary and CIO leadership transitions and project reassessment.6b. Lessons Learned /
Issue / Situation / Lesson /
1. / Senior leadership / The business case for this project was presented in Sept. 2011 by then State Engineer (SE) John D’Antonio and then CIO Renée Martínez as part of the agency’s FY13 IT Plan. / Strong, constant, visible support from the cabinet secretary and his direct reports is required for key, organization-wide improvement efforts such as this project.
2. / Governance / The agency’s Information Technology Governance Committee (ITGC) had reviewed and endorsed the project under SE D’Antonio, but met less frequently and with less direct SE guidance after D’Antonio’s departure.
In retrospect, project scope, complexity, and risks were understated in presentations to the ITGC. / Role and responsibility of the senior governance group must be clear, substantial, and exercised.
The senior governance group must meet regularly, be briefed adequately, and participate actively in project guidance.
3. / Project scope and methodology / Original project scope was to “redesign and modernize” the agency’s “Water Rights Information Management System” using the Appian BPMS and the ESRI GIS platforms. When implemented, the systems would replace the legacy WATERS system, which uses a relational database and 4GL procedure programming.
The project specified implementation in two major phases: domestic water rights processing followed by (2) non-domestic processing. It called for consultants to use agile methods, but the overall approach was a mix of traditional waterfall planning and requirements definition with agile technical specification and programming.
The domestic and non-domestic business processes initially—and incorrectly—were understood to be more distinct and separate than they are. The phased approach turned out to be unworkable and project scope was reduced. The key project deliverable was re-defined to be a BPMS-based application to support Well Driller licensing (Well Driller Application, or WDA).
Subsequent to implementation of WDA, a smaller project was begun to track water rights application status. It, too, was scaled back to focus on processing payment receipts. This project has halted in February 2015 because it was not central to new agency priorities. / The approach described at left attempted to replace the existing legacy system via (a) “big bang” deployment of a new system that (b) sought to embody reengineered processes implemented (c) with a new software platform that differed substantially from software technologies, (d) with which IT staff were unfamiliar and which required outside contractors for software development.
Each of the tasks (a) – (d) above is a substantial challenge in its own right. An incremental approach would have been wiser:
a. The agency should have defined smaller, shorter, projects targeted at a longer-range overall goal to replace the legacy system.
b. It is best practice to reengineer business processes before rather than as part of system development. This practice was not followed; instead, the project attempted to do both concurrently.
c. Prior to such a large project focusing on a core agency system, the agency and staff should have developed experience and expertise via smaller, lower risk projects.
d. Projects focused on core business systems should develop long-term staff capability and capacity to support the systems and not rely on contractors or consultants. Outside contractors were not cost-effective because they did not deeply understand either the BPMS technology or the agency’s unique water rights processes.
4. / The BPMS model / Appian BPMS users have successfully deployed very large systems that manage well-defined business processes with short cycle times. However, it is not clear that it is as well suited for business process such as at OSE/ISC, which are characterized by variable, less consistent, long cycle time processes.
Appian BPMS strengths include (a) process design, modeling; (b) task management and assignment, (c) document production; and (d) interfaces designed for mobile devices
Appian weaknesses include (a) challenges in deploying version releases that implement changes in workflow of long, still-executing processes (b) creating interfaces for desktop rather than mobile devices; and (c) traditional non-process oriented, data entry and updates. / Software architecture and platform capabilities must match high-level business requirements.
OSE water rights-related information systems are fundamentally data-, status-, and report-centric rather than process-centric.
This project does not indicate lack of viability of BPMS-based systems. BPMS-based solutions remain appropriate for agency business requirements that include clearly defined processes, task assignments, and short cycle times.