Good quality data is essential for monitoring and tracking energy and resource consumption. From a single site to large property portfolios, data is aggregated and collected from a variety of sources, including utilities, sub-meters, manual readings and internal systems. With advances in technology it is becoming easier to collect data, but the reliability and completeness of this information is often hidden unless advanced analysis is carried out on a frequent basis.

Common energy data source issues:

• Stale values - an indicator that a meter may have malfunctioned
• Missed file delivery - commonly occurs when wireless/mobile technology is used to extract the data from site on a daily basis
• Data completeness - meter systems are prone to send null values
• Incorrect date/time following clock changes

At EnergyDeck we take this matter very seriously and provide our users with detailed, automated data quality reporting and alerts. Our platform learns the behaviour of each data source and offers complete summaries of inconsistent and missing data to ensure any analysis is carried out with confidence.

Data quality notifications are available for all meter types in EnergyDeck. They are automatically created during data source setup. We classify a data source as any automatic feed (AMR) coming into the platform. This could be a monthly file in CSV format sent by email from a utility provider, or more high-frequency data provided by a Building Management System (BMS).

Data Quality module

EnergyDeck’s Data Quality module is linked to the Data Sources module, monitoring all incoming data into the platform. A high-level alerting tool provides an immediate status of all connected sources (like metering gateways or Data Collectors), whereas the Data Quality view provides more granular information on what the issues are.

A range of checks is automatically run on a continuous basis in the Data Quality module:

• Data source frequency - the system measures the AMR data arrival frequency. After learning the behaviour of the source, it predicts the date and time of the next update. If the update doesn’t occur on schedule, the user is notified
• Meter reading frequency - the system will analyse historical AMR readings and provide a notification if any readings are considered to be missed during a period (e.g. only 44 half-hourly values were delivered out of a potential 48 on a given day)
• Stale readings - there are cases in which a data source and meters send data with the right frequency, but the values are effectively the same. Where data is not being continuously analysed, it is possible that this type of issue can be missed for several weeks, if not months. The data quality module will provide a notification upon receiving any duplicated (stale) files from the data source
• Alerts - creating automatic alerts via email is a central part of the tool. This provides flexible notifications, within an hour of an issue being detected and up to a monthly summary

Notifications are automatically set up as the system begins to study the performance of a data source, providing the user with immediate feedback. Advanced settings allow users to refine the alerting; options include pausing the learning while a data source is commissioned, manually selecting the time of day readings are expected, along with an allowed delay in the expected delivery time.

Metering hardware is notorious for overwriting historical readings. This usually occurs following a communications error, when the meter is still functional but the data cannot be extracted. After a short period of time, typically less than 30 days, the meter memory is then overwritten with the most recent readings. If you are using high-frequency meter data for analysis, financial reporting or fault detection, gaps which can never be filled are frustrating and often unnecessary. Using the EnergyDeck data quality tools, you will be alerted when a meter has missed its expected data delivery date, allowing you to act on the issue before it’s too late.

Meter setup menu

To quickly reference any issues with an individual meter a data quality log can be found within the meter setup tool.

Learn more

If you’d like to learn more about how EnergyDeck can help your business health check its energy data, please get in touch (info@energydeck.com) or explore the rest of the EnergyDeck platform.