Anomaly Detection Report Energiemonitoring Maarten van Someren, Olga Pietras, Tim Leunissen December, 2015
Instituut voor Informatica FNWI, UvA
1
Management samenvatting
Veel energie in gebouwen gaat verloren doordat installaties niet goed functioneren omdat ze niet goed zijn afgesteld, defect zijn of niet goed gebruikt worden. In veel gevallen wordt dit herkend na klachten van gebruikers van het gebouw of bij onderhoud of inspecties. Doel van dit project is om beheerders van gebouwen en andere belangstellenden zoals studenten te ondersteunen bij het vroegtijdig herkennen van problemen met energie-installaties, in sommige gevallen zelfs voordat problemen manifest worden. In het kader van een deelproject is een webservice gebouwd waar gebruikers hun historische gasverbruik data kunnen uploaden en laten analyseren op ongewone waarden. Gebruikers geven ook hun postcode op en daarmee worden weerdata gedownload van het KNMI. Die data worden gebruikt bij het voorspellen van gasgebruik en het herkennen van afwijkingen. Gebruikers kunnen ook de gebruikstijden van het gebouw (bv openingstijden) opgeven en dan wordt ook hiermee rekening gehouden. De webservice plot de data en markeert afwijkende waarden. De gebruiker kan met deze informatie verkeerde instellingen en het begin van storingen van de installatie herkennen. De webservice maakt het zo eenvoudig voor iedereen die historische data kan verzamelen van energiegebruik om aanwijzingen te vinden voor problemen met het energiegebruik van de installatie. In het andere deelproject zijn methoden ge¨ımplementeerd voor het herkennen van afwijkingen in de data van Gebouw Beheers Systemen. De gebruikte aanpak is eenvoudig en algemeen en kan worden toegepast op data van sensoren en actuators (bv kleppen, instellingen van pompen en ventilatoren). Het algorithme zoekt in de data naar afwijkingen van wat normaal is. Afwijkingen die tegelijk of kort na elkaar optreden worden herkend en aan de gebruiker gepresenteerd. Deze kan de data inspecteren en de oorzaak van de afwijking proberen vast te stellen. Naar verwachting maakt de analyse van alle data van Gebouw Beheers Systemen het mogelijk om automatisch een verscheidenheid aan problemen te herkennen. De aanpak met historische data is innovatief. Het is goed mogelijk dat de aanpak met historische data het beheer en onderhoud van installaties sterk zal verbeteren. Volgende stappen zijn het verder uitbouwen van de functionaliteit van de analyse van Gebouw Beheers Systeem data naar een breder scala aan problemen en het integreren hiervan met de visualisatie van de webservice.
1
2
Introduction
There are many data available about buildings that can be used to reduce energy consumption. In this project we address two closely related ideas. The first is to automatically detect deviations in standard energy consumption data and the second is to detect unusual events in more complex data about installations that are collected by Building Management Systems. Originally we planned to develop a software system that students and building managers can use to analyse and monitor energy consumption. However, we realized that this is not practical because we cannot supply support for such a system and that the cost of developing robust software for students is beyond what is possible in the context of this project. We therefore decided to develop a web-based system instead. Students or building managers can upload energy consumption data, their location and opening hours of their building and the system identifies periods with unusual energy consumption and presents these to the user. The system develops a model of the relation between weather conditions, opening hours and energy consumption over time using the energy consumption data and weather data from KNMI. This model is used to identify unusual events. The model is completely data-driven. The other subproject is to extend the detection og anomalies to other components of the installation. We obtained data from installations in a building of the University of Amsterdam and developed a basic anomaly detection system. The data were collected from the Building Management System that controls the installation. Here again we follow a simple purely data-driven approach. This first experiment is based on basic statistical techniques. In this report we summarize the design and function of the webservice and the BMS anomaly detection system, the state of the evaluation and recommendations for next steps.
3 3.1
Energiemonitor Webpage Goal
We want to create an opportunity to find out how basic analytics techniques can be used for predicting and monitoring. This project is mainly aimed at building managers and students. A webpage is a good way to do this since they are easily accessed. Users provide historical data about a building and the task of the system is to identify irregular, anomalous events and display this to the user.
3.2
Website
A live version of the Energiemonitor can be found at ”http://188.166.100.113:3000”. (We will add a domain name). A Dutch manual can be found in the appendix to this report. The starting page can also be found there. Here we give a brief overview of the functionality. Thd user starts by creating an account. Next he can add a new building by specifying the location on a map and entering a name. When a building is created, data needs to be added for the system to learn about how it reacts to different weather condition and response to usage, see 1.
2
Figure 1: Adding a dataset The details of formats can be found in the appendix and on the website. In essence the system needs to know if a building is being used at that time of day, how much power/gas is used and what the weather conditions are at that time. Since weather conditions are normally not stored by the user they are automatically added if not provided. For this the user must enter the postal code of the building. The estimation method is based upon a neural network, as described in (de Nadai & van Someren, 2015). The gas consumption at each 15 minute interval is predicted from temperature and windspeed, gas consumption in the previous period, and the season. A prediction model is constructed from all data. In the next step, time intervals with a much larger or smaller gas consumption than predicted are identified as anomalous and displayed to the user. Figure 2 illustrates the output: a plot of a series of gas consumption data with the anomalies that were detected, indicated by circles.
3
Figure 2: Example of a plot with anomalies
3.3
Technical details
The system is run on a Linux server and written in Node.JS. The storage is done in a MongoDB database. In essence it has three basic tasks: 1. Run the webserver 2. When the user provides data about a office it enhances the data with accurate weather data. 3. When (new) data is provided about a building it retrains itself to estimate the power usage of the building.
3.4
Evaluation
The system was tested on data that were used in earlier experiments, see (de Nadai & van Someren, 2015). The results were similar: predictions about power usage can be accurately made when combining these systems with weather predictions. At the time of writing we have not yet been able to do further evaluation of the system. This evaluation will be done the coming months. The functionality of detecting anomalies in gas consumption data is now made available for everyone. In other studies savings of around 20% in energy consumption are reported using a similar approach. It is not clear yet how general this effect it and which factors determine its size. However, cost is very small and therefore it easily becomes a cost-effective solution. We will keep the webservice running for a year and if possible we will maintain and improve it. We expect that after this period it will become surpassed by on the one hand commercial systems and on the other hand by availability of much wider and stronger functionality, obtained by including more information both from BMS and from other sources of data.
4
4 4.1
Anomaly Detection in GBS data Goal
The purpose of this subproject is to explore basic algorithms for anomaly detection on a wider range of BMS data. In fact, BMS systems may not measure and store gas consumption because a separate system for this is used as the basis for billing. In this subproject we go beyond data on gas consumption and consider all data that are available from a Building Management System. The goal of the analysis is to detect anomalies in the behavior of the climate control system. It is not a common practice to store data for more than a short period during which engineers are solving a specific problem. Fortunately we were able to find historical data from a building of the University of Amsterdam that were collected and stored for a longer period. The results are intended as a proof of concept of anomaly detection in buildings. On the one hand this makes it possible to evaluate the effect of the approach, but at the same time - and at least as important - it serves to illustrate the concept to parties from very different backgrounds like Facility Management, ICT departments, and Energy consultants. At the moment of writing we have only been able to validate the results with our own knowledge of anomalies. We will soon present the results of the analysis to the system experts who collected the data. We will ask them to indicate if they recognize the anomalies that we found.
4.2
Data
The data used for designing and testing the anomaly detection framework is collected from the climate control system in building REC-M of the University of Amsterdam. This building had major problems with temperature regulation during 2014. The span of the data collection is two years: from 1/12/2013 till 30/11/2015. This time span contains enough anomalies that can be detected, but also enough regular data to model normal behavior of the system. Observations are recorded around every 5 minutes, which gives a total of over 100.000 datapoints. The dataset contains seven variables to describe the system: TIN Temperature input, the basic temperature blown into the buildings pipes to distribute among the rooms. The basic temperature is quite stable around 17 degrees. It is further regulated per room by secondary air regulators. TRE Temperature return, the temperature of the air leaving the building. It is used as a measure for the average temperature in the building. WTW Heat wheel, this is an energy conservation system that uses energy from the returned air to heat up the fresh air. The value of this variable depends on the outside temperature. RAW Heat regulator, a heating system that is used in case the heat wheel cannot heat up the air to the desired temperature.
5
RAK Cooling regulator, the air cooling system. Its value depends not only on the outside temperature but also on the humidity level. MME Humidity level of the air after WTW, but before heating or cooling. MRE Humidity return, the humidity level of the air leaving the building. It is used as a measure for the average humidity level inside the building.
4.3
Approach
Anomalies are data that do not conform to a notion of normal behavior. An anomaly detection system therefore spots unusual or unexpected observations. Two types of anomalies are distinguished: 1) a global anomaly is an observation that stands out compared to the whole dataset, 2) a local or contextual anomaly differs from the current pattern and is therefore only anomalous in its specific context (Chandola, Banerjee, & Kumar, 2009). 4.3.1
Gaussian Distribution
A Gaussian - or normal - distribution is used to model the probability distribution of observations that tend to spread around a center value. Rare observations receive a low probability value and can thus be detected as anomalous. The threshold to flag an observation as anomaly is set at a probablility of 0.001. A Gaussian distribution detects global anomalies quite well, however it may fall short in detecting local anomalies. 4.3.2
ARMA
The Auto-Regressie Moving Average (ARMA) process is frequently used to model time series data. In this process a predicted value of a variable is a linear function of a specified number (also called ’order’) of past observations (the AR-part) and past errors (the MA-part) (Zhang, 2003). For example, an AR(1) model predicts a value based on (a linear function of) the previous value. A MA(1) model starts with the overall average of a series and predicts a value by adjusting this with (a linear function of) the previous variation from the mean. As ARMA is based on previous observations, it is more suited to detect local anomalies than the Gaussian distribution. After modelling the data with an ARMA process, the predicted values are subtracted from the original observations, giving a residual. This residual series is then analysed with a probability distribution, as to detect the values with a low probability. Specifically, observations with a difference of more than three standard deviations from the average, are flagged as an anomaly. 4.3.3
Filter
The ARMA model works only well for stationary time-series, meaning a series whose probability distribution is time-invariant (Kalpakis, Gada, & Puttagunta, 2001). Therefore, when the data has a clear trend, this should be filtered out before applying the model. An appropriate technique for this is the HP-filter, which decomposes a series into an overall trend and the residual series. 6
4.4
Results
Different variables have a different normal behaviour pattern, and therefore require a different approach. Temperature in the building is supposed to be centered at a certain constant value, therefore a simple Gaussian distribution is an appropriate model. As the temperature data used is very anomalous for the first few months, these months are excluded in the calculations of the distribution parameters. Figures 3 and 11 show the resulting anomalies in the temperature data. In figure 3 we see the data for Temperature input, the temperature of the air after heating and cooling by the climat control system. The vertical red bars indicate where the system finds anomalies. Some anomalies last for a very short period of one observation (the very light red bars), but others extend over a period of weeks. Most anomalies concern high values over an extended period, like the periods in April and March 2014 with up to 25◦ C. Others concern extended periods with very low values, e.g. 14◦ C in June 2014. These kind of anomalous base temperatures have a negative impact on energy consumption, as the secondary regulators will cool down or heat up to the appropriate room temperature. This regulatory effect can be seen in figure 11, which shows less anomalous data for the same time periods. However, this is only the average temperature in the building, so certain specific rooms are still expected to have some problems during that periods. This is something that should be looked into further in a follow-up project. In any case, a quick detection of anomalous behaviour in the base temperature system, decreases energy consumption as well as problems with uncomfortable indoor climat.
Figure 3: Anomalies in Temperature Input
7
Figure 4: Anomalies in Temperature Return A similar analysis can be done with humidity variables. The inside humidity levels (MRE and MME) depend on the outside humidity level, which leads to a clear annual cycle in the data. This cyclical trend is part of the normal pattern, and thus filtered out with the HP filter before searching for anomalies. The residuals after filtering still show a high variation with both global and local anomalies. To be able to detect the local anomalies, a more complex model than the gaussian distribution in needed. The MA(1) model is the most efficient model that gives accurate results. The Cooling regulator shows a similar pattern as MRE and MME, as it also depends on humidity levels, therefore the same combination of techniques is used. See figures 5, 6, and 7 for the results. The humidity levels show various short sequences of anomalous observations, which sometimes correspond between the two variables. The anomalies in the Cooler are mostly sudden peaks to maximum values during the cold months. Indeed the anomalies correspond to the sudden low temperatures as shown in figure 11. Immediate detection of such peaks could reduce unnecessary air cooling and cold air inflow.
8
Figure 5: Anomalies in humidity before cooling
Figure 6: Anomalies in Humidity Return
9
Figure 7: Anomalies in Cooling Regulator The normal behaviour of the Heat regulator is to only switch on when the heat wheel is fully on (i.e. 100%). All observations where the Heat regulator is on (i.e. above 0%), the Heat wheel values are checked. To allow for a small error, WTW observation below 95% are flagged as anomaly. For the tested dataset this gave no anomalous observations, implying that the Heater has been working correctly. The normal behaviour of the Heat wheel itself, only depends on the outside temperature. This is an external variable that is not included in the scope of this subproject. Further developments of the anomaly detection system should include weather variables to check for Heat wheel anomalies. Overall, several very anomalous but short observations are found, that often correspond among the different variables. Moreover, extended periods of deregulation are detected. The causes of these problem are for now evaluated by system experts, and could in a future project be automated by adding multivariate analyses.
4.5
Conclusions
In this experiment we explored a simple approach to detect anomalies in the energy installations in buildings. We used data that were collected in the context of problems in a building that took place some time ago. Our goal was to show that our approach can re-discover these anomalies in the data. Although we have not yet been able to validate this, we do find clear anomalies. The data, the analysis and the results are in a form that can be presented to persons from varying backgrounds. We expect that this material can therefore be used to explain the approach to various parties and thereby inspire them to invest in additional and more extensive experiments. Directions for further work are to extend the experiments we did to more sets of data, to add to these data gold standard anomalies and to extend the types of anomalies that can be detected. For example, we did not have time to explore an algorithm for detecting trend 10
and changes in trends. Also anomalies may cause a complex range of additional effects on different time scales. There is much to explore here.
5
Concluding remarks
We developed a website for analysing gas consumption data to find anomalous events and a basic system for analyzing data that are typically collected by a Building Management System. A basic evaluation shows that both systems work. Further evaluation is needed to explore strenths and weaknesses in practice. Both systems are useful tools to explain the concept of health monitoring and anomaly detection on buildings to an audience that is not familiar with the technology. They can well be combined into a system that visualizes and analyzes multiple BMS variables. Detailed monitoring and analysis of Building Management System data makes it possible to detect problems with a building and its installations much earlier than current practice which is usually reactive rather than pro-active. Errors in control settings, malfunctioning parts of systems are only detected when s serious problem occurs of when users of the building complain about conditions in the building. Online analysis of the data make it possible to notice problems much earlier and thereby improve maintenance. From an economic viewpoint using the webservice is free. It is difficult to estimate the benefits that consist of less gas consumption but earlier experiments suggest that these are in the order of 10 to 25%. Experiments are needed to provide more evidence. This is also true for the BMS data. Extracting and storing these data needs an infrastructure which involves an investment and maintenance but once the infrastructure is in place the costs are small. Memory is cheap and the basis analyses probably can be automated. Arrival on the Dutch market of several companies (e.g. RedShareT M , EP &T Global) suggests that there is a business case for this technology. We currently focus on demonstrating how the analysis works on existing data of known problems. This should convince interested parties to take part in evaluation experiments which will enable us to obtain a first measure of the effect. The University of Amsterdam has already initiated this process by developing a system for unified storage of data. In collaboration with them we plan to set up a few small experiments with existing data and after that a more full-fledged experiment. Before this it is difficult to predict the effect on energy consumption. Beside better use of installations we expect that better maintenance will also prevent damage to installations which will have a positive effect on waste of materials. The current system will be used the coming months for further experimentation, evaluation and improvement. In January 2016 student projects will extend the range of anomalies that can be found and at the same time we plan to collect a dataset with anomalies that are recognized by human experts to enable further evaluation and development. We plan to continue the collaboration with the Facility Services of the University of Amsterdam to develop an infrastructure for managing energy systems in the university buildings. One of the lessons that we learned during this project is the importance of open standards and availability of data. Although there is a standard for BMS data (BACnet) companies try to protect there interests by introducing small deviations that make the use of components from other suppliers risky or impossible. To enable the innovation and advances illustrated 11
in this project, it is very important that external parties are able to get access to BMS data and to the control settings of BMS systems. This will allow development and testing of new ideas and new technology. Note that the BMS data are normally owned by the user or owner of the building rather than the installation company. In addition to this a uniform way of describing components and processes in installations and more general in buldings is needed for the development of more powerful and yet generally applicable analysis systems. We plan to set up collaboration with the VU Amsterdam to make an inventory of the current situation in the field and the possibility of a uniform ”ontology”. At the same time we hope to work with the Facility Services and ICT departments of the University of Amsterdam to develop a framework for collecting data from different buildings, installations to enable analysis of all data in an efficient way. Another lesson is that privacy is always an issue. In our view, data on the level of buildings are not a threat to the privacy of any person or organization. Yet it took some effort to get permission to use these data. Some data may be a threat to confidentiality or privacy, although this is hard to imagine for the current data. However, collecting data at a detailed level brings with it the possibility to allocate costs to activities at a more detailed level than is currently possible. Also there may be threats to confidentiality of activities. We are interested in collecting data about the use of the building, number of persons present, activities that are relevant to energy consumption (e.g. high-energy laser experiments). A framework and procedure is needed to get permission to collect and use such data when this does not put privacy and confidentiality at risk and to enable parties that can be related to the data to specify for which purpose they allow which data to be used.
Acknowledgment The authors wish to thank SURFSara for funding this work and the Department of Facility Services in the person of Mahnoud Mohamed and LH Techniek in the persons of Jan Hemke and Lloyd Leter for helping us with data and background information.
12
References Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41 (3), 15. de Nadai, M., & van Someren, M. (2015). Short-term anomaly detection in gas consumption through arima and artificial neural network forecast. In Procs. 2015 ieee workshop on environmental, energy, and structural monitoring systems (eesms 2015) (p. 250-255). Kalpakis, K., Gada, D., & Puttagunta, V. (2001). Distance measures for effective clustering of arima time-series. In Data mining, 2001. icdm 2001, proceedings ieee international conference on (pp. 273–280). Zhang, G. P. (2003). Time series forecasting using a hybrid arima and neural network model. Neurocomputing, 50 , 159–175.
13
APPENDIX: Manual webservice (in Dutch) Het volgende bevat belangrijke informatie over het gebruik van de website en een technisch overzicht over hoe deze werkt. Gezien de website in een opstart fase is kan het dat deze niet altijd naar behoren functioneert.
Handleiding Aanmelden Voor het aanmelden van een gebouw ga je simpelweg naar ’188.166.100.113:3000’ en voer je inloggegevens in. Als geregistreerde gebruiker kun je ook het voorbeeldpand CWI bekijken waarvan de data regelmatig wordt ingelezen.
Figure 8: Huidige inlog / registratie formulier Bij het toevoegen van een pand wordt je gevraagd om de naam en de locatie. De locatie wordt gebruikt om een zo goed mogelijke inschatting te maken van de weersomstandigheden gedurende het jaar.
14
Figure 9: Toevoegen van een pand Invoer Het systeem gebruikt historische data van het pand om een inschatting te maken van hoe het er het jaar erop eruit ziet. De historische data kan in CSV (comma, tab or semicolon seperated) of JSON formaat worden ingelezen. Daarbij moeten de volgende hoofdlettergevoelige velden verplicht aanwezig zijn in iedere datapunt/regel: • timestamp Een tijdsafdruk die leesbaar is voor de date.parse functie in javascript. De beschrijving hiervan is te vinden op tools.ietf.org. Let hierbij op dat het om de einddatum van het tijdsvak moet gaan om consistent te zijn met de manier waarop de temperatuur wordt berekend. Het timestamp wordt tevens omlaag naar het dichtstbijzijnde uur afgerond (e.g. 15:12 wordt 15:00). • gas
15
Het verbruik van het pand in m3 in het afgelopen tijdsvak. • power Het verbruik van het pand in kWh in het afgelopen tijdsvak. Met de kantnotitie dat ofwel gas ofwel power kan worden weggelaten. Daarnaast kunnen de waardes wat betreft de weersomstandigheden worden overschreven door de velden “windspeed” en “temperature” aan elk datapunt toe te voegen. • usage (niet verplicht) Kan waarde 0 of 1 bevatten en geeft aan of het gebouw in gebruik was in het tijdslot. Wanneer niet beschikbaar wordt uitgegaan van normale openingstijden; maandag tot en met vrijdag van 09:00 tot 17:00. • windspeed (niet verplicht) De gemiddelde windsnelheid in m/s in het afgelopen tijdslot. • temperatuur (niet verplicht) De gemiddelde temperatuur in graden Celsius in het afgelopen tijdslot.
Voor de opmaak van nummers dient rekening gehouden met dat ook deze van een “string” in een daadwerkelijk waarde dienen worden omgezet. Dat betekent dat deze geen spaties of extra punten kan bevatten. Meer informatie over de parseFloat functie is te lezen op deze website: www.ecma-international.org. Aanpassen van het tijdsinterval Het kan zijn dat uw data niet gemeten zijn met het tijdsinterval van een uur dat het systeem veronderstelt. Er zijn twee situaties: uw data zijn met hogere of met een lagere frekwentie gemeten. Hogere frekwentie: verdeel de data in intervallen van tijdsduur L; bereken het gemiddelde van het interval en maak hiermee een nieuwe tabel met de gewenste intervallen. Lagere frekwentie: in dit geval is de enige oplossing het kopi¨eren van datapunten. Als u bijvoorbeeld elke twee uur informatie krijgt kopieert u elke regel en past u het tijdstip aan.
16
Uitvoer Het systeem produceert in eerste instantie een plot van de data. Hieraan kunt u zien of de data goed ge¨ınterpreteerd is. Vervolgens wordt via het KNMI relevante weerdata opgehaald. Dit gebeurt per ongeveer 100 datapunten per keer en duurt tot 250ms duren. Wanneer deze data beschikbaar is wordt deze aan de grafiek toegevoegd.
Figure 10: Normal view of the data before anomalies are detected In de laatste stap wordt een neuraal netwerk getraind om waardes te kunnen voorspellen. Indien dit gebeurt is staat dit aangegeven. Het model zal nu: Voorspellingen geven over het gebruik op dit moment (gebaseerd op weer en ingeschatte opening van het gebouw). Aangeven welke waardes in het verleden als anomalie¨en worden beschouwd.
17
Figure 11: Anomalies
Technische beschrijving Hoe wordt het voorspellingsmodel uit de data gehaald Wanneer de data beschikbaar is wordt in de eerste stap datapunten aangemaakt waarop het algoritme gaat leren. Deze datapunten zijn verschillend van de datapunten die aangeleverd worden. Een datapunt bevat behalve informatie over de huidige situatie ook informatie van het moment ervoor en erna. Al deze datapunten gebruikt het algoritme om te leren een voorspelling te maken hoeveel stroom er wordt gebruikt. Dit “leren” kun je voorstellen als het beginnen met een volledig willekeurig model en stapgewijs onderdelen daarvan aan te wijzen die heel goed werken of wat minder goed werken. Herhaal dit vaak genoeg en een model “leert” dat warm weer zorgt dat er minder gas gebruikt hoeft te worden en visa versa. In de software ontwikkeld voor SURFSARA wordt dit leren gedaan met een neuraal netwerk. Wat meer in detail worden eerst de waarden genormaliseerd d.m.v. het geven van een Z-Score. Hiermee worden alle waarden gecomprimeerd tot een waarde tussen 0 en 1. Het vervolgens gebruikte neurale netwerk heet “Brain.JS”. Wat is een anomalie Hier zijn verscheidene defenities voor. Het model dat gebruikt wordt op de webservice detecteert enkel punt-anomalien. Dit betekent een afwijking van de voorspelde waarde met meer dan 15 procent. Hiermee zou je dingen kunnen detecteren als het erg lang open staan van een raam in je huis maar lange termijn problemen worden niet gedetecteerd. E.g. het langzaam groeien van het stroomverbruik door een slijtende CV-Ketel.
18
Data bewaren Het systeem bewaart de data na uw laatste inlog gedurende twaalf maanden. Daarna worden ze verwijderd.
19
