ERCOT Market Operations DPO
Event Description: MarkeTrak Lessons Learned – Business Processes Focus Group / Date: Wednesday, January 10, 2007 / Completed by: J. Lavas
Attendees: Karen Malkey (CNP), Debbie McKeever (TXUED), Kristi Tyra (TXUED), Johnny Robertson (TXUE), Shannon Bowling (Cirro), Cheryl Franklin (AEP), Blake Gross (AEP), Kathy Scott (CNP), Kyle Patrick (RRI), Kris Brown (Constellation), Rob Bevil (GnMntn), Jennifer Garcia (Direct), Brad Treatch (FCP), Trey Henry (Stream), Daryl Everett (ADS), Michael Matlock (Gexa), BJ Flowers (TXUE), Marcie Zlotnik (StarTex), Mindy Seaton (Suez)
ERCOT Attendees: Adam Martinez (MODPO), Justin Rasberry (MODPO), Scott Egger (MODPO), Jamie Lavas (MODPO), Mike Taylor (MarkeTrak Development), Sherri Slagowski (Testing), Dave Michelsen (RCC), Jack Adams (RCS), Farrah Litton (RCS), Richard Gruber
Phone Attendees: Jane Eyanson (AEP), Bryan Haddock (TNMP), Jeff Keifer (Reliant), Rachael Burns (Direct), Monique Patillo (TXUE)
Summary of Event
1.  Note worthy items:
·  Phase II Requirements to be tabled for future discussions.
·  Met Tuesday, January 9 for the technology focus group. Today to be centered around on business processes. Major themes to be identified and discusses at the RMS February 14, 2007 meeting.
·  Review with group the high level project life cycle.
·  Notes and discussion were broken down by project phase. Listing what went well and what areas needed further improvement.
2.  Initiation & Planning
A. Business Requirements:
What went well?
·  ERCOT attended multiple on site locations and meetings – DM
·  Info gathering was good and Market Participants were able to review and synch up after the initial info gathering sessions – KP
What areas need improvement?
·  The entire process carried on for too long. Would like to see a long term timeline for phase II – DM.
·  Vender selection to gap analysis was a considerable amount of time. Serena was selected in the summer and gap analysis documentation was not given until the beginning of the next year. Did not get to incorporate funds that would have been needed for missed requirements/unavailable functions into the budget – SB
·  Did not receive an itemized list and review of the list for items that were not included and why they were not included – DM & KP
·  JR – ERCOT recognizes that Market Participants would like to be included in the vendor selection process. However, as long as the requirements are being met Market Participants should not be involved in the selection processes. But gap analysis should be communicated more timely.
·  Market Participants did meet with ercot.com on the vendor selection process and felt this was beneficial since they were able to see the product and demos. ERCOT could take measures to protect names of companies so that they are not revealed and Market Participants feel that ERCOT choose a tool that benefited ERCOT only – DM
·  Several session were held where Market Participants rated and prioritized features and feel that some of these features were still excluded – MP
·  ERCOT staff helped explain that once Serena was selected as the vendor, this information was brought back to Market Participants and conveyed that this vendor could best meet the financial needs and requirements as ERCOT had no internal knowledge of the tool. It took time to bring on contractors to get ramped up with the tool.
·  Once a contract is negotiated, it may be helpful to have the vendor meet with a group of Market Participants to reduce assumptions and misunderstandings. Even if business requirements are locked down and this is a “scoping session” to discuss what is meant would be helpful – BJF
·  In agreement especially since TDSPs and CRs often take different approaches and views – KM
·  JR – ERCOT delivery team should participate in a requirements review with the body that set the requirements – LL Theme
·  Need to practice the old fashion K.I.S.S theory in order to keep things simple. Especially since we are looking at code being written multiple times in many instances. Also need to practice getting something in and stable before we move on – BJF
·  Market Participants are paying for their implementations and there were system dependencies that existed at ERCOT. If this was known, this may have changed the view. This ultimately impacted scope, time and budget. We should better weigh the pros and cons. Also, the timeline was dictated to the market and led to having the cart before the horse – SB
·  JR – goes back to having a steering committee that could review the schedule and make determinations from there that could mitigate issues of this type.
·  AM – We liked BJs suggestion. Shannon’s points bear merit but would like to remind the group that the technical dependencies are out of the project teams control. IE – Tibco solution. MarkeTrak happened to be one of the first projects to come out since the conversion to Tibco. We should have better resolved the risks and an IA could have alleviated this.
·  Still feel like we have missing requirements that are a big issue – DM
B. Analysis & Design
·  JR – Based on discussions yesterday feel that what is needed is not the conceptual design but the need for detailed design applicable for MP needs including integration. I.e. - data elements for the API.
Execution and Controlling
C.  Development:
What went well?
·  Have heard very few complaints on the GUI side. Business kept going as usual. Usage is high. On the API side, there were too many assumptions made which lead to issues. Once technical people were able to get on line together these issues were resolved in less than an hour.
What areas need improvement?
·  JR – another identified theme from yesterday was getting the right IT people engaged at the right time - earlier. Need better schedule management to make this happen.
·  When Market Participants were still analyzing the documentation to gain an understanding of what was provided, ERCOT was already providing an implementation date to RMS. This timeline for implementation should not start from the time ERCOT is clear on the requirements but from the time the market is clear on the requirements and design – DM
·  AM – ERCOT was working towards an informally agreed upon timeline. We worked back form the requested 6 months from publication of design docs to establish that time. That was the initial date. That date was not final and moved several times which accounted for some of the delays in the timeline. It was discussed at RMS and feedback was always solicited.
·  The feedback was RMS was in agreement with ERCOT since ERCOT was committed to making the dates. The Market wants a timeline that will take Market Participant timelines into consideration not just ERCOT – DM
·  AM – again, calls were held where Market Participants also committed to making a date.
·  Market Participants felt pressured where there were budget increases. Market Participants were not inclined to state they were not able to meet the date and be a showstopper for everyone. Another LL for other project – if we have API and GUI, why do they need to go in together? The API held up the GUI. A steering committee could have helped to drive these decisions – SB
·  BJF stated that Market Participants did not want to write processes twice which would have been the case for everyone using the API.
·  A WG/steering committee would be much more beneficial than just an email and reviewing requirements separately. Also, with getting the API in, CNP was able to extremely streamline processes – KM
·  KP asked for clarification on that benefit. Were the improvements with the tool itself or being able to use an API? KM stated the API was the reason they were able to automate. KP stated they were trying to gain a better understanding because they are creating reports and still ramping up. KM agreed that using the API for this has been very beneficial. KP agreed as they are trying to get ahead of the learning curve to utilize the reports more efficiently.
·  Report capability training was the only piece of the training that was lacking. Would have liked to see a listing of field names etc. – JE AM asked what type of documentation was provided around a listing of terms? Scott stated that there was definitely documentation lacking in that area. There is not a field by field listing of all available terms/fields. (Scott Egger has since followed up with direction on where to obtain this information as it was available in the User’s Guide)
·  Suggestion for getting some MPs together for some code sharing so that everyone does not need to reinvent the wheel – BJF
·  JR – If Market Participants would like a forum to share the code, doc, deliverable – the steering committee would be most appropriate.
·  Issue with the vendors not sharing information other than informal conf calls when the API users got together to hash issues out – DM
·  Agreement as there are some inconsistencies – JE
·  Did like that there was follow up training once the date was pushed. Felt it was a lot easier to implement with the review – SB & KP
·  Also helped those that went to both sessions to have the experience twice to start familiarizing things. Especially those trainings combined with the sandbox training – KP
·  Could not use sandbox for training because it was being used for API testing. Would like to see an additional environment so that people can use it for training purposes as needed – SB
·  User Guide…would be nice to have a link to the user guide on the tool instead of having to find it on the web – JE
·  AM – since the project will be closing we have moved the information to another location. This needs to be communicated to the market.
D.  Design Changes:
·  Meeting at TXUED with Scott and Dale without an SME present was not beneficial – DM
·  Whose decision was it to break out DUNS numbers within the same umbrella? – CF. Scott explained that it had to do with using the DC for security purposes.
·  How many times did we demo where we were with the project before training? – KP. Scott explained that the first demo was just screen shots, but the second one was a true product demonstration. There were technical difficulties during the product demonstration. Would like to have something like that in the future either localized or done through web ex – KP. LL: better schedule management and communication to include more demos.
·  SE – Maybe focus on future training needs up front in requirements gathering so that it can be accounted for and implemented into the timeline which could also account for more than one day of training.
·  Announced during training that the functionality changed from a one to many format to a one to one format which came as a surprise – MP. Scott state that this came from getting rid of the spreadsheets which was communicated. KP noted they were ok with the change and it worked well for them although it didn’t for many others. Scott explained that you can not determine the status of the ESIID without it being a one to many format. MP stated that the change was not a bad thing just the timing was bad.
·  One of the requirements was that they did not lose the one to many format and they did. They had to build an API and a workflow piece to account for this. Now understand this but it needed to be communicated better – DM. Another example of communication misunderstanding in the interpretation – BJF. Remembered the vendor stating that was the case and it was something ERCOT had to digest and accommodate as well – KP
·  Mike Taylor stated he was fairly certain this was discussed at RMS and from that came the whole idea of grouping tickets. AM stated it may have been discussed but it was the wrong audience.
·  Maybe along with separating out the demos, can separate out the DEV cutover for issues from the DTD issues to help keep it simple while you are stabilizing and getting accustomed to the new product. Understand this put some burden and extra requirements on other system groups but feels that it outweighs a bad or misunderstood roll out of a new product – BJF
·  To ensure everyone is hearing and seeing the same thing, it may be helpful to discuss issues in the context of the tool – SB. Scott stated this was a great idea but adds time and cost for prototyping.
E.  Testing:
·  Recapped issues discussed from yesterday.
·  Testing of the GUI was discussed at TTPT and at RMS and voted down. Feel that they are their own coach and that they are aware that other utilities have encountered issues where some Market Participants are turning to them for some tutoring on the tool – KP
·  From an educational standpoint, requiring testing may help get others used to using the tool which would have been beneficial – AM
·  Felt as if testing scenarios were created with ERCOTs assumptions and conditions. Would like to have ERCOT reach out to the market for this in the future as opposed to using a simulated CR – KS
F.  Go-Live & Implementation:
·  JR – Noted that ERCOT needed to maintain the existing application during the cutover which was discussed at the technology focus group. Also captured was that once we are in prod and have post prod issues…Market Participants are limited to testing in prod with their data. Would ask that we maintain a test environment for them to utilize as needed.