By Mary Rich and Irshad James
Analytics is not new to the utility industry. Utilities have been using the data from their financial, billing and usage, SCADA (Supervisory Control and Data Acquisition), DMS (Distribution Management System), GIS (Geographic Information System) and OMS (Outage Management System) to produce reports and do analytics for years. What is new is the amount of granular data they have from smart grid.
So how has the smart grid changed this for the utility?
- Real time or near real time data
- Much more accurate data
- Huge amount of data
- Many entities now want the data
- Reporting moving out of IT to the client sector
- Customer wanting access to the data.
The list goes on and on. But all these changes present a lot of challenges to the utility that have not really been addressed.
There are several challenges in dealing with this large influx of data.
- Size – In IT for most utilities the largest databases were in the financial applications. The smart grid databases have tripled and sometimes quadrupled in size. IT struggles to ensure that these databases are monitored and sized properly while trying to put systems and processes in place to do the backup and archiving of this data.
- Duplication of data – data is being sent to several applications in the utility and this presents a problem with keeping the data in sync.
- Data extracts are being requested from these large databases. We need to identify who handles them, whether they have an effect on the performance of the database and who approves these requests.
- Use of extract, transform and load (ETL) tools to extract and transform the data for other sources.
- Reporting and ensuring this is not detrimental to the performance of the databases.
- Leveraging data experts on the data models and the entity relationships.
- Working with data architects that are constantly tuning the performance of the database.
Many utilities have turned to moving data into a data warehouse. This is probably what most utilities will have to do to keep the duplication down and deal with reporting on these large volumes of data. However, this also presents its own set of challenges:
- Is the data complete or is there a need to go to multiple systems to pull the data?
- Are there standards in the data being placed in the warehouse?
- Is the data real time for some and outdated for others?
- Is there governance around the warehouse?
- Who has access to the data?
- Who are the subject matter experts (SMEs) on the data?
- Is there a tool interfacing with the warehouse that makes it user accessible?
- Security – who has access and to what data?
It is important that the utility starts small and works smart. There needs to be strong governance around the data. There must be standards in place that ensure any data moved into the warehouse follow those standards. The data should be moved in manageable chunks, ensuring the presence of complete data. There needs to be a user friendly tool with security to let the clients get to the data. It is important to have statements on reports if they are not going to be official reports. Data analysts or business analysts need to be available that know the data and help users with reporting. Users must be trained on what information is in the warehouse and its benefits. Issues must be addressed one at a time. It is important to learn and adapt.
If reporting is done by the systems of record and those same reports are generated from a data warehouse, it is important to ensure that the results are the same. Otherwise there could be huge regulatory ramifications for the utility.
In most cases the clients that support the individual systems, for example outage, are very familiar with the data. They have canned reports that they run. They know the rules around the reports, and for years they have been what the utility bases its CAIDI and SAIDI on.
The utility could end up with multiple systems with different information with no knowledge on what is counted as an outage or not counted, pulling and publishing reports in the company from different data sources.
All that being said about the struggles with the data, there are many things that the utility can really benefit from by using this data.
There are several key players in the meter data analytics sector. A brief overview of each of these competitors is presented here. The decision to choose the appropriate vendor depends on several factors, such as compatibility with currently invested hardware and software, pricing and specific goals. Cloud-based analytics could help solve the ‘big data’ issues and concerns but do nothing to come up with the use cases for analytics.
eMeter Analytics Foundation1
eMeter (Siemens), with its meter data management (MDM) product EnergyIP, has been helping utilities collect, archive, validate, estimate, edit and process meter data for several years. eMeter Analytics Foundation for EnergyIP offers utilities develop insights into usage patterns and other information from raw data. eMeter Analytics Foundation helps leverage the raw data into meaningful charts, graphs and diagnostic reports. It uses ETL tools to retrieve data from the EnergyIP MDM and load it into a special analytics database, which is based on a star schema. This makes it possible to quickly obtain analytic information such as aggregates, means, medians, etc., rather than just obtaining transactional information such as the usage of a particular customer during a particular period of time. The eMeter Analytics Foundation runs in conjunction with the eMeter Reporting Framework, and also provides a collection of standard, ready-to-use reports.
Oracle Utilities Meter Data Analytics2
Using eight pre-configured dashboards, Oracle Utilities Meter Data Analytics helps utilities improve metering performance and analyze trends in energy consumption. The event dashboard facilitates tamper event monitoring, and helps protect revenue by identifying tamper events that need investigation and rectification. The ability to drill back to Oracle Utilities Meter Data Management allows for the checking of device history for any prior revenue protection issues for this device. Further revenue protection is achieved by helping utilities identify non-functioning or malfunctioning meters, identify common factors, and rectify them. The consumption dashboard also helps utilities protect the revenue by identifying low usage customers, and then drilling back into meter data management to compare current and historical consumption.
SAP Smart Meter Analytics3
SAP Smart Meter Analytics addresses the challenges of increasing the effectiveness of demand side management programs, complying with regulatory targets, generating revenue and reducing energy costs, improving accuracy of load forecasting, and reducing churn rates in deregulated markets. Key features include instant aggregation of data and analysis of customer energy usage at any level of granularity, aggregation and dimension, precisely segmenting customers based on consumption patterns, comparing energy usage of customers against peers, using root cause analysis to improve their energy efficiency, and empowering customers with self-service access to energy usage insights.
IBM Smarter Analytics for Utilities4
IBM Smarter Analytics for Utilities aims to better manage the large influx of ‘big data’ coming from smart grids and meters, by effectively using past history of weather, loads and environmental factors to predict outages and track demand patterns. Key benefits include improving generation performance, transforming the grid from a rigid analog system to a dynamic and automated energy delivery system, empowering consumers by providing them with near real time, detailed information about their energy usage and meeting greenhouse gas emissions targets while maintaining sufficient cost effective power supply.