Processes for logger data acquisition and quality characteristics

Explanation

The positions of the different processes in the workflow of the logger component can be seen here in the graphics.

I Data transmission

The raw data (level 0 data) automatically collected by the logger are either sent directly to the DMP via mobile radio (HTTP interface) or temporarily stored on an (S-) FTP server where they are cyclically read in by the DMP. Alternatively, if the logger does not provide for the ability to send data automatically or historical data is to be acquired, Level 0 data can also be collected manually via the DMP web interface.

II Saving raw data

Once Level 0 data is captured in the DMP, it is backed up in a complex database system. This database system is intensively maintained to enable very high availability and guarantees regular backup of the data. Further processing of the data is non-destructive, so that in case of doubt it is always possible to fall back on the backed-up raw data.

III Reading in raw data

Parsers are implemented in the DMP that interpret the individual raw data formats and normalize the data into a uniform structure. When the data is extracted, it is linked to the metadata defined in the sensor configuration. In this process step, the measured values can be converted using measurement functions or subjected to a rough plausibility check using limits.

IV Quality assurance

During quality assurance, quality characteristics (flags) are assigned to the individual measured values. Only the quality of the recorded Level 1 data is evaluated. Any manipulation of the data is not intended. Quality assurance can be performed manually or automatically. The characteristics provided so far can be found under quality flags.

V.1 Importing reimport data

Using automatic quality assurance results in reimport files, which are read in by the DMP in a similar way to the Level 0 data. Unlike the raw data, the format is uniformly fixed and can be viewed under manual upload.

V.2 Linking of Level 2 data and quality characteristics

The quality assurance and, if necessary, the re-import of the quality characteristics result in the Level 2 data.

VI Provision of the data

The Level 2 data can also be made available outside the DMP for presentation, publication and further processing via various interfaces. For example, it is possible to provide the data via a sensor observation service (SOS) or to offer simple web interfaces.