Background
Utility providers deal with a lot of data. The volume is large, the variety is vast, and the velocity is in micro-batches all the way to real-time.
If utility companies can harness the power of their data, it becomes a competitive advantage and differentiator. Given the nature of the services that utility companies provide, these data-driven improvements may have a far greater impact on customers beyond monetary value.
For utility providers, the business initiatives are significant:
Moving to time-of-use billing
Billing customers more accurately
Incentivizing times of consumption during dormant periods
Monitoring green initiatives such as public electric vehicles and the impact they have on the network
The first challenge for Orlando Utilities Commission (OUC) was centralizing huge IoT-style datasets for analysis.
Challenge
Keyrus partnered with the Meter Data Operations (MDO) team at OUC to build a centralized data warehouse to serve as a foundation for their 5-year utilities data analytics roadmap. A centralized data warehouse enables the commission’s downstream teams and technologies to run the right analyses to answer questions they’ve been asking for years but couldn’t previously answer. Questions like:
What is the overall reliability of our network?
How can I become more proactive and reduce the number of estimated billings due to meters not producing reads?
How can I better use voltage profiles to identify mismapped meters?
The utilities commission asked the Keyrus team to capture the output coming from the AMI/AMR meters in the field as the primary data. Subsets of the data existed in different places, including proprietary systems, which made it extremely challenging to address the team’s requirements.
The only other way to consume the entirety of this information was to parse through the individual XML files produced by the head-end system. XML presents challenges even in small data volumes, and the commission was receiving 2,000 files every day!
Since OUC had never been able to see and analyze all of this data before, there were dozens of business questions to consider. We prioritized the following business use cases to direct the technical development of our centralized data warehouse, or Meter Data Management Platform:
Network Connectivity (electric and water)
Voltage Pattern Analysis
Data Quality
Negative Reads and No-Reads
Solution
Working closely with the MDO team, Keyrus created an entire Meter Analytics Reporting Kit (MARK) that would answer the questions of today and serve as a bedrock for the questions of tomorrow. With the IoT nature of the business, the platform needed to ingest, transform, store, and consume over 100 million rows per day in a performant, cost-effective way. We selected best-in-class, cloud-friendly technologies including Talend for ETL/ELT, Snowflake as the data warehouse, and Alteryx and Tableau as the downstream consumption tools.
Unless rolled-up and delivered in meaningful visuals, analyzing 15+ billion rows would yield meaningless results. Our platform included user-friendly dashboards to make these analyses accessible to a wider audience. It also aggregated very large sets of detailed information into simple visuals that clearly illustrate outliers or risks (e.g., missing reads, meters mapped to incorrect transformers, etc.).
With these visuals, the utilities commission can drill-down and filter their data to the lowest granularity, not only providing the “what” to their business questions, but also the “why.”
The entirety of the platform was supported and hosted through AWS. With a focus on DevOps and security, the architecture was deployed via CloudFormation templates to ensure standardization and governance around the different services leveraged to support the platform. We ensured data was only accessible to the consumers who had the right permissions by using the principle of least privilege and creating secure routes for the different components to talk to each other (i.e., PrivateLink between Snowflake and the customers’ VPC).
Data Governance
Because this type of information is highly coveted but often misused, the Keyrus team supported data governance alongside the technical solution. This support included:
Defining the ownership and organizational roles within the utilities commission needed to support the platform
Conducting impact analysis for end users
Formalizing, prioritizing, and capturing requests for the platform
Assuring data quality procedures
Results
The Meter Data Operations team is now using a centralized data platform to heavily expedite the daily ad-hoc requests they get from across the organization. A request for registered reads or intervals used to take weeks to pull together, depending on the requirements, but now only takes minutes.
The dashboards we built provide proactive views into faulty meters and allow for the MDO team to create field activities for agents to go out and fix the equipment. With more agents proactive in field activities, the reliability of the network improves, which reduces estimated costs, and increases customer satisfaction. Other dashboards help improve the accuracy of meter-to-transformer mapping by showing distance outliers and mismatched voltage patterns. The MDO team now supports the greater utilities commission mission to reduce outage times and provide exceptional reliability.
With a centralized data warehouse, the utilities commission can now leverage their information for high ROI initiatives such as fraud detection, leak prevention, usage profiles for customer segmentation, transformer overload, and momentary outage analysis.
The Meter Analytics Reporting Kit empowers the utilities commission along their 5-year strategic data analytics roadmap to positively affect their bottom line, increase customer satisfaction, and improve reliability in providing water and electricity to over 200,000 customers.
Ready to get started with your own Meter Analytics Reporting Kit? Contact us.
Hear from the Team
Watch the video below to hear Kevin Maroney, Senior Director of Delivery, talk through an overview of the work we did with OUC.