ETV Protocol Outline ==================== DHD20160413 PRIMARY METRIC -------------- M1) Show reduced slope of kWh/HDD for primary household heating fuel use when OpenTRV is installed and enabled, indicating increased heating efficiency (and reduced waste / carbon). Control can be one of: C1) Previous heating bills vs historical HDD. C2) OpenTRV sensors near existing rads not fitted with OpenTRV valve heads. C3) OpenTRV valve heads with most of the smarts permanently disabled. C4) OpenTRV valve with most of the smarts only enabled after a delay, not announced to the user. C4 is our preferred control. C1 is least likely to work amongst fuel poor households (eg prepaid). It may also be useful to try to extract per-radiator savings estimates. SECONDARY SUPPORTING METRICS ---------------------------- M2a) Show increased R^2 (correlation) for kWH/HDDi for primary household heating fuel to show better control of heating fuel wrt to weather. M2b) Show better room temperature stability ("T|C16") during occupancy. M2c) Show lower room temperature stability ("T|C16", "tT|C") during occupancy. M2d) Show effective room automatic temperature setbacks ("tS|C") during vacancy. TRACEABILITY ------------ T1) Show traceability of HDD back to METAR data. T2) Show traceability of heating fuel kWh back via supply meter. T3) Note claimed accuracy of OpenTRV temperature sensor devices, and note assumption that relative temperature changes rather than absolute values are more important to the energy and carbon savings. INPUTS ------ All inputs to be timestamped and archived to allow non-realtime analysis of the data (and audit) and logged with the timestamp and appropriate units, and preferably in an easy-to-understand (eg non-binary) format. Timestamps should by default be in UTC unless otherwise justified. Timestamps should be of an appropriate granularity/precision for the stamped datum, typically at least: pr1) to the minute or hour for air temperature p2r) to the hour or day for energy consumption p3r) to the day for HDD Where a data representation with format with variable precision is used, eg ISO 8601, by default is should be arranged that there is useful information in the least significant digit shown, and preferrably an absolute error of no more that 1 or 2 units in the last place (ulp) unless otherwise stated. Where appropriate, and particularly for the non-public data sets, data should be protected against transmission errors and eavesdropping at least. All sensors and heat controllers must have a unique ID to which data points and control outputs are unambiguously tied. I1) Some basic descriptive data on each household should be provided such as rough geographical location (eg "South east London, UK, 3 bedroom detached house, solid wall 1930s, family of 4, social housing, some fuel poverty issues") but not in form that allows (re-)identification of the household or occupants. I2) Timestamped room air temperature readings gathered regularly with a target precision and absolute accuracy of better than 1C at least hourly, and permantly stored for analysis, and associated with a given heat source controller and zone. a) This is typically the T|C16 measure for OpenTRV devices, in units (and default precision) of 1/16 degrees C, sampled every minute and typically transmitted wirelessly every 4 or 8 minutes depending on other data to be sent. b) Temperature measurement can be taken at the heating device control, eg in the OpenTRV valve for C3/C4, or at a separate sensor unit in the same room / heating zone for C2, but in any case the relationship between the heat controller (eg radiator valve) and the sensor must be clear, such as coming from the same unit with a unique ID or tied by unique IDs of controller and sensor. (It is possible that users will physically move separate sensors to the wrong zone by accident at least temporarily.) Temperature measurements can be taken at the controller and one or more separate sensors, but in this case a primary sensor should be designated, or an algoritihm for synthesising such primary temperature value from the sensors. c) It will be useful to note the location vertically and relative to the human occupancts of the temperature sensor(s), eg close to ground level near the heat source for sensor readings collected from OpenTRV all-in-one radiator valves. I3) Heating controller actions and auxiliary data such as detected occupancy hould be timestamped and permanently recorded against the controller ID, eg setbacks applied, actuator operations/values (such as valve open %). I4) Heating Degree Data for the local area for each household gathered at least daily and generally no less than weekly, with appropriate timestamps to match up with (I2), with a demonstrated link back to METAR or equivalent, eg using historically matched points. a) Unless there is a good reason otherwise, a default baseline HDD temperature of 15.5C will be used. I5) Space heating energy consumption will be measured by reference to the household supply meter (traceable back to calibration standards), with before and after checks of any auxiliary reading devices (eg to transmit the data live and in fine grain) against this meter. This meter could be domestic gas or electricity or heat supply. This should be sampled and timestamped at least weekly, preferably daily or hourly. At least 8 distict daily samples are required in each period of operation (eg full vs reduced/control). a) The supply used for heat can be used for other purposes also, such as cooking and hot water; the linear-fit algorithm is expected to be able to eliminate most of the effects of those if enough data points are taken, especially though variable weather and/or out of the heating season. b) If the household uses significant other secondary heating then it may have to be discounted from the results. I6) Depending on the control/baselining method used, there may need to be a period during which most of all of the automatic energy saving features are turned off, making them act as if a more simple version of the control directly following manual instruction. a) It must be possible to know when specific heat controllers have reduced or full automated savings engaged, eg based on time or on data sent by the devices such as an explicit status flag/datum or inferrable from others. I7) Interventions made during the test period should be recorded, eg substitution of failed equipment or additions/removals of heat controllers (eg adding OpenTRV valves to more radiators). a) If any of this auxiliary data would allow (re-)identification of the household (ie present a privacy issue) then it should be separated into discloseable and non-discloseable sets and possibly an reduced non-re-indentifying version of the latter added to the former. Where that is not reasonably possible, and audits cannot be conducted confidentially, a household may have to be removed from the data set. PROCEDURE OVERVIEW ------------------ P1) Install heat controller devices and sensors, timestamp and permanently record data for subsequent access and analysis, eg on an internal store or in real-time over wireless. a) Precision and update frequency of data should be as stated under inputs. P2) Gather local Heating Degree Day (HDD) data or raw temperature data, and archive with timestamps suitable to match against heat measurements. P3) Gather household space-heating kWh demand/usage and archive with timestamps suitable to match against heat measurements. P4) Take baseline measurements of energy meters and other parameters, and details of household structure and occupancy. P5) Incrementally during the test period, and at the end of the test period, perform some or all of these interim measures: a) Check calibration of remote metering devices against base meters. b) Note intervations/changes in test household. c) Note switches of devices between full savings and restricted mode as appropriate. Compute kWh/HDD values in the different regions. d) Perform interim calculations of kWh/HDD against controls. These calculations are to use the line-fitting regression test as per IPMVP 2002 (http://www.nrel.gov/docs/fy02osti/31505.pdf) and 2012 standards. e) Make site visits to inspect state of equipment, maintain, and without asking leading questions, gauge user response to system. P6) Preserve data for audit. P7) Publish anonymised data and results as appropriate. TODO ---- Traceability of the energy usage measurements is key and that comparison of the loop meter and supply meter measurements is good measurement practice. Re: use of industry best practice, particularly for baseline definition. Key documents are: International Performance Measurement & Verification Protocol http://www.nrel.gov/docs/fy02osti/31505.pdf http://evo-world.org/en/products-services-mainmenu-en/products-ipmvp-mainmenu-en ISO 50015:2014 Energy management systems — Measurement and verification of energy performance of organizations — General principles and guidance ISO 50006:2014 Energy management systems -- Measuring energy performance using energy baselines (EnB) and energy performance indicators (EnPI) -- General principles and guidance Evaluation Methods. This will involve statistical analysis and must include measurement uncertainty (a requirement of the ETV GVP and ISO 17025). A guidance document is under review by the ETV project technical working group with the following general principles: * The final claim (verified performance) has to reflect the conclusions of the statistical analysis * Uncertainty to be provided in the most appropriate form. * The independence of parameters should be investigated. * A "design of experiment" is advised in case multiple operating parameters do influence the performance. When this is not possible, a minimal set of relevant measurement points representative of the operating range has to be taken into consideration. This is to avoid testing the technology's performance exclusively in favourable conditions that are not representative of the full range of operations. * Exclusion of outliers: due justification for setting some data aside needs to be provided. All collected data are to be presented and discussed. Partial or selective elimination of data is not allowed unless there is a good reason for doing so and this should be transparently acknowledged and justified. In the case of abnormalities in test data for which no explanation is found, the tests may have to be redone. * Caveats have to be reported transparently. The protocol should be written so that each of these points can be addressed.