Skip to content

Manual Upload

Explanation

This page explains the different import and reimport formats. Since most formats expect a quality flag, it is helpful to have a look at the different quality flags first.

Technical framework

  • Fields for values (e.g. "AirTemperature2maGAvg10min" or "value") have a maximum length of 200 characters.
  • Fields for comments (e.g. "quality_comment,") have a maximum length of 1000 characters
  • Fields without a value are either empty, contain spaces or the string "NIL"

Example: All rows contain a valid zero value in the third column:

timestamp, AirTemperature2maGAvg10min, SurfacewaterConcentration1mbSMax1min, BatteryVoltage
2013-09-12 12:23:33, 20.3,, 11.7
2013-09-12 12:23:33, 20.3,    , 11.7
2013-09-12 12:23:33, 20.3, NIL, 11.7
  • Individual files must not exceed a maximum length of 100,000 lines
  • Individual files must not exceed a maximum size of 10 megabytes

Manual Level 0 Upload

Description

Upload or import of raw data for loggers that cannot send data to the DMP automatically or data taken from the logger manually.

Specification

  • Usually CSV with date and time field
  • Table header with field labels is recommended

Specification data fields

  • Data structure corresponds to the one configured in the logger component

Example

timestamp, AirTemperature2maGAvg10min, SurfacewaterConcentration1mbSMax1min, BatteryVoltage
2013-09-12 12:23:33, 20.3, 8.21, 11.7
2013-09-12 12:43:33, -273.1, 43.1, 8.6

Manual Level 2a Upload

Description

Upload or reimport of quality flags without further processed data in ASCII-CSV format. The reimport must be preceded by a data export because the measured value ID is required as a key for subsequent assignment to the raw data.

Specification

  • Table header with field labels is required

Specification data fields

  1. Field level1_value_id (position 1, mandatory).
    • Contains measurement value ID exported from database
    • Mandatory to get reference to measured values
  2. Field quality_flag (position 2, mandatory)
    • Contains the quality flag ( OK, DOUBTFUL or BAD )
    • Mandatory required
  3. Field quality_cause (position 3, conditionally mandatory)
    • Predefined reason (e.g. BATTERY_LOW) or OTHER
    • Mandatory if quality_flag not OK
    • List of predefined reasons to be maintained by TERENO data managers
  4. Field quality_comment (position 4, conditionally mandatory)
    • Free text field for comment or extended description of quality_cause
    • Mandatory if quality_cause matches OTHER

Example

level1_value_id, quality_flag, quality_cause, quality_comment
23443, OK, ,
23444, DOUBTFUL, BATTERY_LOW, ,
23445, BAD, OTHER, Signal cable broken

Manual Level 2b Upload

Description

Upload or reimport of quality-assured data and their quality flags. Required if data is recorded or further processed externally. Before further processing, a data export is recommended in order to obtain, if possible, the measured value ID as a key and thus the reference to the raw data.

Specification

  • A dot (".") is to be used as decimal separator
  • For zero values the character string NIL is to be used
  • One measured value and its quality flags are described per line
  • Table header with field designations required

Specification data fields

  1. Field timestamp (position 1, mandatory).
    • Contains date and time when the measured value was recorded
    • Format YYYY-MM-DD hh:mm:ss
  2. Field sensor name (position 2, mandatory)
    • Name of the sensor, results from level 2 sensor configuration in the logger component
  3. Field value (position 3, mandatory)
    • Measured value or NIL in case of non-existent/incorrect values
  4. Field quality_flag (position 3, mandatory)
    • Contains the quality flag ( OK, DOUBTFUL or BAD )
    • Mandatory
  5. Field quality_cause (position 4, conditionally mandatory)
    • Predefined reason (e.g. BATTERY_LOW) or OTHER
    • Mandatory if quality_flag is not OK
    • List of predefined reasons to be maintained by TERENO data managers
  6. Field quality_comment (position 5, conditionally mandatory)
    • Free text field for comment or extended description of quality_cause
    • Mandatory if quality_cause corresponds to OTHER
  7. Field level1_value_id (position 6, optional)
    • Contains measurement value ID exported from database
    • Recommended to get reference to measured values

Example

timestamp, sensorname, value, quality_flag, quality_cause, quality_comment, level1_value_id
2013-09-12 12:23:33, AirTemperature2maGAvg10min, 20.3, OK, NIL, NIL, 23144
2013-09-12 12:23:33, SurfacewaterConcentration1mbSMax1min, 8.21, OK, NIL, NIL, 23145
2013-09-12 12:43:33, AirTemperature2maGAvg10min, -273.1, BAD, Other, Signal cable broken, 23146
2013-09-12 12:43:33, SurfacewaterConcentration1mbSMax1min, 43.1, DOUPTFUL, BATTERY_LOW, NIL, 23147

Further information

  • empty fields at the end can be omitted (quality_cause, quality_comment, level1_value_id)
  • already recorded values will be overwritten
  • identification by unique sensor name and timestamp
  • if a timestamp occurs in two different l2raw records, the one parsed last wins