Smart Emus and Smart Meters

Smart Emus and Smart Meters

A few months ago, we applied to Innovate UK’s Energy Game Changer competition, which sought to disrupt the energy sector by using novel and innovative techniques from small companies based in other industries. We are very proud to announce that we were successful in this application and have been awarded project co-funding by Innovate UK – the UK government’s innovation agency!

At Emu Analytics we focus on real time processing and analysis of high volume, high velocity data. As many of the issues found in the mobile industry (where we have great expertise) are similar to those found in the energy sector we thought we were well placed to tackle such issues. We will be applying our knowledge of big data, real time processing, data forecasting and machine learning to create insightful, innovative and commercially valuable insights from the processing of smart meter data sets. We aim to build a data processing, analysis and visualisation technology stack that will explore aggregated metrics of energy usage.

We have kicked off the project by establishing some relationships with organisations within the energy sector, to ensure that our understanding of smart meter data is complete. We are therefore delighted that we have managed to get one of the UK’s “big six” energy firms on-board already, SSE plc. Their Chief Data Officer, Dan Robertson, stated:

“SSE are constantly looking to innovate with data and analytics to improve our service to customers and to reduce the country’s carbon footprint. We are excited to be working with Emu Analytics to look at innovative uses of Smart Meter data that could help meet those goals” - Dan Robertson, Chief Data Officer, SSE

We have additionally established ties with Delta Energy & Environment, a specialist consultancy working in the energy sector, who will also help us promote the innovative opportunities of the project to the energy sector. Jon Slowe, Director at Delta-ee adds the following:

“Smart meter technology is beginning to yield a huge amount of data. It has the potential to reveal how households and businesses consume and interact with energy, in unprecedented detail. These insights could be used to drive improvements in a range of areas, from energy customer experience, to power infrastructure, and even the promotion of energy efficiency objectives. The key to unlocking these insights and driving these opportunities, will lie in the development of innovative analytical solutions.” - Jon Slowe, Director, Delta-ee

There are many similarities between the energy and mobile sectors and we feel that we are perfectly placed to transfer our knowledge and apply it to smart meter data.

For those of you who don’t know much about them, here is a quick background to smart meters…

A smart meter is the next generation of electricity meters. They show you how much energy you are using and report back directly to the energy company, removing the need for estimated bills or meter readings. They give near real time information on how much energy is being used and how much it is costing, allowing for better energy management and budgeting. Before, information on energy consumption was only collected every 6 months, but with smart meters, it will be collected at ~30 minute intervals. The EU’s Third Package Directive requires member states to ensure than 80% of households have a smart meter by 2020. In the UK this will mean about 53 million smart meters will be fitted in homes and small businesses. Following the recent election, the new Smart Meters Bill was highlighted in the Queen’s Speech, which continues to push the smart meter agenda, even if not within the same boundaries as the EU directive.

Some of our key metrics will allow:

  • Comparison of energy use by consumer type/industry/building type/town/region across a variety of temporal scales.
  • Forecasting of energy use at various temporal and spatial scales
  • Identification of outliers
  • Identification of erroneous, faulty or inefficient consumption patterns
  • Segmentation of the population based on energy usage characteristics
  • Classification of building type based on energy usage characteristics

So What?

As with any project we tackle the “So what?” is important. Its all well and good analysing and aggregating all of this data but who is going to want to use it afterwards? Well, there are a few areas that we have identified that could benefit from these aggregated insights, this is certainly not an exhaustive list and we are sure there are many other interesting use cases (feel free to let us know if you have any great ideas!).

Emission Reduction Schemes

Smart meters can be used to reduce energy use by putting the user in a more informed position of how much energy they are using and how much it is costing. By looking at the aggregated energy use over different temporal and spatial scales, houses or regions that are using higher amounts of electricity in comparison to the ‘norm’ can be identified. Education schemes/advertising/insulation installation/low energy initiatives could be targeted to these areas and by comparing current and historical datasets the impact of such schemes can be assessed. The data could also be invaluable to sustainability departments of large regeneration schemes as they can see if they are having a positive impact on the average amount of energy use in an area by implementing their more sustainable and lower emission infrastructures.

Demographic Classifications

People and buildings can be classified into groups based on energy use. The data could also be used to update existing demographic segmentation tools e.g. the census based Output Area Classification. Household electricity consumption has been shown to be influenced by factors such as household type, demographic type, number of occupants and socio-economic status.

Supply/Demand Management and Flexible Pricing

Increased reporting, aggregation and displaying of energy usage data would also allow more accurate prediction of peaks and troughs. Resulting in reduced wastage, reduced need for energy storage, reduced potential for outages, increased efficiency of the system and more confident trading of energy. This would all help to drive down costs for the energy companies and the users. By identifying peaks and troughs, flexible pricing can also be introduced to try and even out energy usage more evenly across the day. For example, energy prices could be lowered during known troughs (at night) and raised during known peaks (after work). Flexible pricing could also be tailored to specific geographical areas.

Planning and Land Use

By analysing energy use patterns, vacant and unoccupied homes could potentially be identified. Housing composition could also be approximated giving a better idea of areas of the country with large amounts of vacant housing and places where people are much more densely packed. This could help councils etc have a better understanding of their populations on a much more granular level than purely relying on the census. Areas with a high proportion of secondary homes/holiday homes could also be identified based on the erratic nature of energy use e.g. low through the week and high on the weekends or low for several months and high through the summer. Machine learning can be used to improve these assumptions by using the data to continually improve the model.

Our metrics could be used to estimate the energy performance of new buildings based on similar buildings which already exist. The predicted energy performance can then be compared with the actual energy performance to improve future building designs.

Currently, wide area energy models (e.g of regions or cities) use aggregated ‘typical’ energy consumption patterns. This doesn’t account for geographical variation and so, models built using actual energy data are likely to be a lot more accurate. These will play a large role in better informing residential energy efficiency policies in the future. Amy Beasley, District Heating Technical Project Manager at the Greater Manchester Combined Authority said:

“Having access to actual consumption data gives a far more accurate picture of what is going on on the ground, enabling local authorities to not only plan and target interventions and investments more effectively and intelligently, but also better measure the impact that energy efficiency projects are having for residents.” - Amy Beasley, District Heating Technical Project Manager, Greater Manchester Combined Authority

Critical Infrastructure

Many large companies have infrastructure that covers wide geographical areas. Based on our knowledge of the mobile sector we know that cell towers could greatly benefit from smart meter data. Erratic or highly variable energy patterns often suggest that there is a fault or problem with the tower and so by identifying these potential faults early, with the use of smart meter data, they can be fixed reducing energy wastage and outages.

Fraud Detection and Avoidance

Smart meters are fitted with encryption and anti fraud measures but this doesn’t make them totally immune from hacking and tampering. By viewing aggregated insights, anomalies can be spotted which could potentially point out areas where fraudulent use of energy supplies is occurring. For example, if readings are abnormally low someone may be fraudulently recording their data, if the readings are abnormally high someone may have hacked onto a neighbours supply. Theft of energy has always been a significant problem and some estimates believe that worldwide electricity theft amounts to almost $90bn.

Large Scale Energy Usage Behaviour Surveys

Traditionally these are expensive to conduct but our analytics dashboards will have a country wide (or at least energy provider wide) view of energy usage across a variety of temporal scales.

Vulnerable Population Monitoring

Elderly and vulnerable populations can be monitored remotely based on energy consumption patterns. If these patterns deviate from the norm e.g. very low energy use all day when normally there are small peaks every few hours then triggers could be set of to suggest that they may need checking on.

Renewable Energy Optimisation

Renewable energies are providing an increasingly large proportion of our energy supplies but energy suppliers have little understanding of how network instability can be affected by altering power flows. By combining smart meter data with historical data and weather data a better forecast of the supply of renewable resources can be attained. In order to forecast the demand factors such as demographics, current consumption, maximum demand and seasonal variations need to be considered.

EPC Data

We hope to include Energy Performance Certificate (EPC) data in this project. The EU directive on the performance of buildings requires energy certificates for any building that has been built, sold or let since 2008. Whilst not all buildings in the UK are covered however, there are still over 15.6m domestic EPCs and over 475,000 non domestic EPCs available to view and download. The dataset contains a wealth of information about energy performance, emissions, potential performance and building attributes such as roof type, window type, building type, number of habitable rooms and tenure type. All of these attributes can be added into our models to understand more about they influence energy use across spatial and temporal scales.

To showcase some of the insights that can be derived from this data, here are a few basic graphs using data from the OX1 and OX2 postcode districts.

The graph below looks at the minimum, average and maximum energy consumption of buildings based on their window type. There is a small overall reduction between pre 2002 double glazing and post 2002 double glazing, interestingly single glazed windows have a lower maximum energy consumption than double glazing, however the average and minimum values are much higher. There is a stark drop in energy consumption for buildings fitted with triple glazing. Obviously, this is not necessarily a causal relationship and there are probably many other factors at play.

EPC Data - Energy Consumption vs Window Type

EPC ratings go on a scale from A to G with A being the best and G being the worst performing properties. These two graphs compare EPC scores and building type - it has been split into OX1 and OX2 postcode districts. There are no A-ratings in OX1 and only a very few in OX2 (19 == 0.2%). In both districts flats have a higher percentage of B ratings than any other housing type, with maisonettes and houses having the highest percentage of G ratings.

EPC Rating by Building Type

The map below shows the most common EPC rating for each postcode sector (e.g OX2 8). No postcode sector had an A, F or G rating as the most common, the majority of sectors were dominated by D ratings. The lowest rated sector was OX1 3 which is in Central Oxford.

Most Common EPC Rating - Oxfordshire

The maximum, minimum and average energy consumption values can also be plotted for each postcode sector. This shows lots of spatial variation. Some have negative minimum energy consumption values which suggest they either pass energy back to the grid from renewable sources or there was a fault with the reading. OX1 4 has the highest average consumption values with OX2 9 having the lowest.

Min, Max and Average Energy Consumption - Oxfordshire


Aside from the techniques that will go into processing the data (which we will cover in another blog) a large amount of effort will be needed to ensure that the design and visualisation of the analytics interface is highly engaging, intuitive and makes the insights immediately accessible to the user. At Emu we have lots of experience building such dashboards, the example below shows building heights for the whole of England. Our experience has shown that the more engaging and intuitive the dashboard is the more likely people will interact with and explore the data and help draw their own conclusions and insights.

Emu Analytics - Building Heights of England Portal

And Finally…

We have also recently hatched a new Emu, Bliss, who will be heavily involved in this project. Bliss is on a placement year from Queen Mary University where she is studying for an MSc in Big Data Science. She brings with her a wealth of experience in all things data science as well as lots of enthusiasm and ideas. Welcome to the mob!

energy, data, opendata, InnovateUK, Smartmeters / by