Section 3.
In the last four years, the Hetch Hetchy Enterprise has spent over $600,000 on the purchase and implementation of Vista and Aces analytical software and on the Data Mart data warehouse, and has committed significant staff and consultant resources on software implementation. However, to date the software programs have not been successfully implemented and there is considerable uncertainty regarding the timeline and additional cost to fully implement short range and long range planning, and water release and electricity scheduling tools. Delayed implementation of the Hetch Hetchy Enterprise's Data Mart, which will compile wholesale and retail electricity meter reading and billing data from different sources, hinders the Hetch Hetchy Enterprise's ability to accurately reconcile electricity bills with the Pacific Gas and Electric Company's electricity meter data. Further, these delays increase the risk of an adverse settlement in the $28 million dispute with the Pacific Gas and Electric Company, in which the Hetch Hetchy Enterprise has challenged PG&E meter data from 2000 through 2003 to determine whether meter usage and payments to PG&E had been correctly computed. Costly delays in Hetch Hetchy's efforts to implement software solutions for its most pressing analytical needs will likely continue unless clear responsibility is assigned to senior managers, implementation timelines are established, and key milestone accomplishments are monitored by the Assistant General Manager of Operations. Without successful implementation of these critical software tools, management will be less able to manage core utility functions or avoid many risks that are inherent to utility enterprises.
The Need for Analytical Tools to Manage the Hetch Hetchy Enterprise
The Hetch Hetchy Enterprise operations require analytical software for long range and short range planning and for daily operations. For the past four years, the Hetch Hetchy Enterprise has been implementing new software tools but the process has been slow and the Hetch Hetchy Enterprise continues to lack the necessary software for planning and for daily operations.
Delays in Implementing Long Range Planning Tools
The Hetch Hetchy Enterprise needs analytical software for modeling the water system and long range planning of the water supply. In the 1990s, the Hetch Hetchy Enterprise, through an outside consultant, developed the Hetch Hetchy Simulation Model, which simulated the Hetch Hetchy system, including the Tuolumne River flow under various hydrologic and weather conditions. The Hetch Hetchy Enterprise also used the Local Simulation Model for long range planning of the local Bay Area water system. In 2000, the Hetch Hetchy Enterprise Water Resources Manager prepared an evaluation of the existing analytical software, including the Hetch Hetchy and Local Simulation Models, and recommended new and upgraded software to replace the existing tools.
The evaluation, entitled HHWP Operations Planning: Modeling and Operations Support System Upgrade, recommended implementation of Oasis, which was a generalized modeling software for water resource systems, to replace the Hetch Hetchy and Local Simulation Models. Because the Hetch Hetchy and Local Simulation Models were not readily usable by Hetch Hetchy Enterprise staff, the Hetch Hetchy Enterprise did not continue to use these models, although Oasis, which had been recommended as an alternative, was not implemented.
In 2002 the Hetch Hetchy Enterprise and Water Supply and Treatment Division staff implemented a more formal process to evaluate and select software for hydrologic modeling and long range planning. A committee was formed, consisting of representatives from the Public Utilities Commission Planning Bureau, the Financial Services and Information Technology Divisions, the Hetch Hetchy Enterprise, and Water Supply and Treatment Division. The committee developed criteria and a plan for reviewing and selecting new software. In June of 2003, the committee recommended to the Public Utilities Commission's Information Technology Advisory Committee that the Hetch Hetchy Enterprise and the Water Supply and Treatment Division jointly purchase a new planning model, called "Mike Basin". The purchase order for Mike Basin was finally approved in June of 2004, after a year-long process of evaluating Mike Basin. Mike Basin is being implemented in two phases. Phase one will model the Hetch Hetchy upcountry water system. Staff training on the software application for phase one has been scheduled for December of 2004. Phase two will develop a system-wide model of the total water system and is expected to be completed in the summer of 2005.
Resource Optimization and Electric Schedule Formatting Tools
The 2000 Hetch Hetchy Enterprise Operations Planning evaluation recommended implementation of two software products for scheduling electricity on the State's electricity grid:
- Vista is a software package that models the hydroelectric system, including inflows to the reservoirs, electricity loads, contracts, and other system constraints. The Vista system is intended to support optimal scheduling of water release and hydroelectric generation.
- Aces is a tool to schedule electricity with the California Independent System Operation, who manages the State's electricity grid.
The Hetch Hetchy Enterprise purchased Vista and Aces in June of 2001. Neither Vista nor Aces has been fully implemented, although the Hetch Hetchy Enterprise has spent $486,185 on these software packages to date. Most of the basic planning and scheduling modules in the Vista software package have been completed although some modeling issues remain outstanding. According to Hetch Hetchy Enterprise staff, when the contract to purchase Vista was approved in 2001, neither Hetch Hetchy Enterprise nor Vista staff were aware of many of the difficulties in adapting specific requirements of the Hetch Hetchy system to the Vista model, indicating that the Vista model had not been fully evaluated prior to its purchase.
Also, according to Hetch Hetchy Enterprise staff, although implementation of Aces, which formats electricity schedules to be provided to the Pacific Gas and Electric Company to schedule on the State's electricity grid, was adequately implemented in 2001, changes to the Public Utilities Commission's agreements with the Modesto and Turlock Irrigation Districts have required changes to the Aces software.
Integration of Vista and Aces has also been difficult. Hetch Hetchy Enterprise staff have worked out an interim procedure to communicate schedule information from Aces to Vista and back, although the interim procedure has been complicated due to the different treatment of Daylight Savings Time by the two software systems. Currently, outside consultants have been engaged to develop a communication procedure between Aces and Vista but the communication procedure has not yet been produced.
In the fall of 2003, a protocol for scheduling with Aces and Vista was developed with plans to train staff. Because of the complexity of the protocol, the Hetch Hetchy Enterprise staff chose not to implement the protocol. Rather, in the fall of 2004, the Hetch Hetchy Enterprise staff is proposing an alternative protocol, in which the Vista software will produce an initial, optimized pre-schedule for scheduling electricity on the State's grid but another tool, similar to an updated form of a Lotus spreadsheet, will be used to produce adjusted pre-schedules and real time schedules. During this process, the Hetch Hetchy Enterprise staff have continued to use legacy Lotus spreadsheets for scheduling.
Implementing Timelines and Accountability for Going Forward
The initial process of selecting new software for long range planning and scheduling has contributed to the delays in implementing the software. An individual manager, with skill in computer modeling, evaluated and recommended the long range planning and scheduling software packages with minimal participation from other Public Utilities Commission staff. The Information Technology Services Business Applications Development Manager stated that he was uncomfortable with the selection of Vista, which had been the recommendation of one individual, but signed the purchase order, acknowledging the Information Technology Services Division could support the software application.
Although the Hetch Hetchy Enterprise and Information Technology Services Division staff began implementing the scheduling software, Vista and Aces, based upon the 2000 Hetch Hetchy Enterprise Operations Planning evaluation, Information Technology Services Division staff initiated a new process for selecting new long range planning software to model the Hetch Hetchy system. An interdepartmental committee was established in 2002 that included staff within the Public Utilities Commission who would use the long range planning model. The committee conducted a needs assessment, researched possible software products, interviewed other utilities, interviewed and evaluated vendors, and called independent users of the software products. The committee process was lengthy, however, and one year passed between the identification of the long range planning software, Mike Basin, in June of 2003 and the purchase order in June of 2004. The Public Utilities Commission staff anticipates completion of the first phase of implementing the Mike Basin software in December of 2004 and the second phase in the summer of 2005.
The process of selecting and implementing the scheduling software packages, Vista and Aces, has been problematic. The software selection was outside of an inclusive evaluation and recommendation process. The initial implementation of Aces was on schedule but changes in the Public Utilities Commission agreements with the Modesto and Turlock Irrigation Districts required remodeling of the Aces software that has not yet been completed. Implementation of Vista has been problematic from the beginning. According to the Hetch Hetchy staff, initial problems in the Vista software were resolved but contributed to delays in implementation. However, Vista software has also not been well structured for the specific needs to the Hetch Hetchy Enterprise system, including modeling of reservoir spills, maintaining required fish flows downstream from the O'Shaughnessy Dam, and other specific issues.
Hetch Hetchy Enterprise staff with day to day responsibility for implementing the Vista and Aces software packages have also been assigned to other high priority tasks. Two consultants have been hired to assist in implementation of the Vista and Aces software packages but have not worked full time on implementation. Hetch Hetchy Enterprise staff anticipate that the consultants will be able to dedicate more time to software implementation in the future.
Going forward, firm timelines and management accountability need to be established to ensure timely implementation of the Mike Basin, Vista, and Aces software. The Assistant General Manager, Operations, should assign responsibility for overseeing implementation of the software packages to appropriate senior level managers, including setting firm timelines and deliverables, and reporting back to the Assistant General Manager, Operations on a regular basis.
Delays in Implementing the Data Mart
The Public Utilities Commission Information Technology Services Division has been implementing a Public Utilities Commission-wide data warehouse that will combine data from the disparate information systems within the organization into a shared database. As part of the Public Utilities Commission data warehouse, the Hetch Hetchy Enterprise is developing a data mart to compile wholesale and retail electricity meter reading and billing data from different sources.
The Pacific Gas and Electric Company provides Hetch Hetchy Enterprise customer electric meter read data to the Hetch Hetchy Enterprise and bills monthly for electricity and related services provided by the Pacific Gas and Electric Company. The Hetch Hetchy Enterprise staff review the Pacific Gas and Electric Company meter data to verify that customer usage and demand charges are correct. Currently, the Hetch Hetchy Enterprise staff verify Pacific Gas and Electric Company wholesale bills against the Hetch Hetchy Enterprise's available data but the base data is not comprehensive, and identifying discrepancies in the Pacific Gas and Electric Company bills has been time consuming.
Verification of Pacific Gas and Electric Company bills is especially significant in light of the Public Utilities Commission's $28 million dispute with the Pacific Gas and Electric Company for charges under the Interconnection Agreement between July of 2000 and December of 2003. In response to the dispute, the Hetch Hetchy Enterprise's Retail Services Section is reviewing three years of invoices and retail meter data1 provided by the Pacific Gas and Electric Company in order to validate and correct the data. The validated retail data is then reconciled with the wholesale data to determine what actual payments to the Pacific Gas and Electric Company should have been. During this three-year period, the Hetch Hetchy Enterprise reviewed and corrected the meter read data for billing purposes but reconciliation of the wholesale bill was not routinely performed. According to Retail Services staff, the Hetch Hetchy Enterprise will validate and correct the Pacific Gas and Electric Company's meter data monthly going forward. However, until the data mart is implemented, the necessary billing data will not be compiled in one place, increasing the time required to validate the data and the chance for errors.
The Hetch Hetchy Enterprise data mart would hold all historical meter data, facilitating the Hetch Hetchy Enterprise's procedures for validating the Pacific Gas and Electric Company's meter data and tying the data into the Pacific Gas and Electric Company's billings.
The Hetch Hetchy Enterprise has a task order to implement the data mart with a consultant, ICF Resources, Inc., who was selected by Information Technology Services Division. The goals of the data mart implementation include:
- Creating an automated retail and wholesale bill reconciliation process.
- Improving the quality of data used for forecasting municipal electricity loads by applying processes developed for wholesale bill reconciliation to an historical database.
- Integrating the data warehouse with the Public Utilities Commissions' other systems.
Responsibility for ensuring implementation of the Hetch Hetchy Enterprise data mart has not been clearly assigned. The task order with the consultant was signed by the Hetch Hetchy Enterprise Acting Director for Power Operations but the Hetch Hetchy Enterprise staff have not been actively managing the consultant. The Director of Information Technology Services and the Hetch Hetchy Enterprise Director for Power Operations should prepare a joint timeline for implementing the Hetch Hetchy Enterprise data mart and regularly review the data mart implementation to ensure that the timelines are met.
Conclusions
The Hetch Hetchy Enterprise, which is responsible for forecasting water supply requirements and balancing water supply and the production of hydroelectric power, lacks the necessary analytical software to perform its core functions. Currently, much of the Hetch Hetchy Enterprise's long range and daily planning are performed on legacy Lotus spreadsheets. Although the Hetch Hetchy Enterprise has purchased analytical software for planning and daily operations, the Hetch Hetchy Enterprise has failed to dedicate the necessary resources to the software implementation. The process of implementing the analytical software has dragged on for four years and is not yet complete. The Assistant General Manager, Operations, should assign management responsibility and accountability to the appropriate senior managers to ensure adequate and timely implementation of the necessary analytical software.
Recommendations
The Assistant General Manager, Operations, should:
3.1 Assign responsibility for overseeing implementation of the software packages to appropriate senior level managers, including setting firm timelines and deliverables, and reporting back to the Assistant General Manager, Operations on a regular basis.
The Director of Information Technology Services and the Hetch Hetchy Enterprise Director of Power Operations should:
3.2 Jointly prepare and enforce timelines for implementing the Hetch Hetchy Enterprise data mart, including regularly reviewing the data mart implementation to ensure that the timelines are met and reporting to the Assistant General Manager, Operations.
Costs and Benefits
The proposed recommendations can be implemented within the Public Utilities Commission's existing resources. Assigning management accountability, dedicating necessary resources, and rigorously enforcing implementation timelines would aid the Hetch Hetchy Enterprise in implementing the necessary analytical software to conduct its core business, resulting in increased efficiency and reduced risk in hydroelectric power production. The Hetch Hetchy Enterprise has already spent $614,432, including $486,185 for Vista and Aces and $128,247 for the data mart, which will be wasted costs without full implementation of these software tools. Further, the Hetch Hetchy Enterprise is reviewing Pacific Gas and Electric Company meter data from 2000 through 2003 to determine what were correct meter usage and payments to the Pacific Gas and Electric Company as part of the $28 million dispute with the Pacific Gas and Electric Company. Implementing the data mart, which will compile wholesale and retail electricity meter reading and billing data from different sources, will facilitate accurate reconciliation of electricity bills with the Pacific Gas and Electric Company's electricity meter data.
1 The Pacific Gas and Electric Company owns most of the retail electricity meters that serve San Francisco's municipal customers. The Pacific Gas and Electric Company reads these meters and provides meter reading files to the Hetch Hetchy Enterprise's Retail Services Section monthly for billing the Hetch Hetchy Enterprise's retail customers.