Vlaanderen.be

Onderzoeksoutput

Whip: Human and machine-readable specifications for data

Onderzoeksoutput: Bijdrage aan congresPaper/Powerpoint/AbstractOnderzoek

Auteurs

Afdelingen, onderzoeksgroepen en diensten

Details

Originele taal-2Engels
Pagina's87
Aantal pagina's1
StatusGepubliceerd - 24-sep-2018
Event10th International Conference on Ecological Informatics: Translating Ecological Data into Knowledge and Decisions in a Rapidly Changing World. - Friedrich-Schiller-Universität Jena, Jena, Duitsland
Duur: 24-sep-201828-sep-2018
https://icei2018.uni-jena.de/

Congres

Congres10th International Conference on Ecological Informatics
Verkorte titelICEI2018
LandDuitsland
StadJena
Periode24/09/1828/09/18
Internet adres

Abstract

Different tools and technologies are available to clean and harmonize data. Independent of the tool used, the ability to assess the quality of a data set and identify potential errors is crucial for harmonization efforts. The necessity becomes even more apparent in the context of data publication, (re)use and aggregation.
Documentation and guidelines about the data requirements provide guidance in this process and enable to communicate what to expect from the data, but are mostly intended for humans only. To facilitate the harmonization process, we propose the usage of a specification file, describing the constraints to which the data should comply. Its syntax is human- and machine-readable, so it can be used to communicate expected data quality/conformity and to validate data automatically. The scope of the set of specifications can be specific to a dataset, researcher or research community, which allows bottom-up and top-down adoption. As an example, we apply the specifications to verify data mapped to the biodiversity information standard Darwin Core.
In this talk, we will present "whip", a proposed syntax and format to express data specifications. Whip allows to define column-based constraints for tabular (tidy) data with a number of rules. We will also demonstrate a software application (called "pywhip") to validate data sets using these specifications. We hope it will trigger a discussion on how to express data specifications and communicate data quality expectations.

Thematische lijst

EWI Biomedische wetenschappen

Technologisch

Vrije trefwoorden

  • data quality
Projecten
Onderzoeksoutput (gerelateerd via auteurs)
Projecten (gerelateerd via auteurs)
Winkelwagen
Toevoegen aan winkelwagen Opgeslagen in winkelwagen

Kopieer de tekst uit dit veld...

Documenten

Documenten

  • whip_abstract

    !!Final published version, 81 KB, PDF-document

    Licentie: !!Unspecified

Relaties
Bekijk grafiek van relaties