# D15: Datasets and data processing #

## Sensor data sets ##
- Time stamp in ISO 8601 format with time zone
- Globally unique ID for the sensor (this can be a combination of a
  concentrator ID + sensor number or any other value that is globally unique)
- Value
- Unit (this may be sent in initial frame only or following request from
  data platform)
- Frame number (used to identify missing frames, sequential number that can
  potentially loop, in which case we need to identify the looping logic to
  ensure missing frame detection works at the loop point)

## Data processing on the platform ##
1. Check that the frame can be parsed, if not return an error (HTTP ???).
2. Check the sensor ID, if known fetch sensor meta-data, otherwise assume
   new sensor. The platform may auto-create the new sensor or send an error
   code back depending on internal logic. Note that this internal logic can
   also depend on how the sensors have been commissioned.
3. If existing sensor, check that frame number received = last frame + 1.
4. If unit is specified check that it matches the unit known for the sensor.
   If unit mismatch, send back hard error code.
5. Store the value and send return code:
   - All OK: HTTP 200 or equivalent
   - Step 3 shows missing frame: non-critical error code asking for missing
     frames, specifying last received frame + last received time stamp.
   - Step 4 no unit stored against sensor and none provided: non-critical
     error asking for unit to be sent in next frame.
     Note that we need to have a unit value for unit-less numbers.

See SenML for unit values. We may want to extend on what SenML provide but we
should also be compatible with it.

## Data processing of return codes on the device ##
1. If all OK, stop processing.
2. If missing frame code, send missing frames in one or multiple messages.
   If some of the frames are no longer available, send a frame specifying
   "unknown" for those.
3. If unit required code, mark the sensor as needing to send unit on next frame.

## Commissioning ##
To be done in D38.