Learn

Open and collaborative data, explained

Plain-language guides to the ideas behind an open and collaborative data platform: what open data is, how teams work on data together, and how to share it safely.

Foundations

What is a data platform?

A data platform is the set of tools that lets an organisation collect, store, organise, share and analyse data in one connected place.

What is collaborative data?

Collaborative data work means many people can find, contribute to, review and reuse the same datasets, with a shared history of changes.

Open data

What is open data?

Open data is data that anyone can freely access, use, modify and share, for any purpose, subject at most to attribution and share-alike requirements.

Open data licences explained

A guide to common open data licences: Creative Commons CC0 and CC BY, the Open Data Commons licences (PDDL, ODC-By, ODbL) and how to choose.

The FAIR data principles

FAIR stands for Findable, Accessible, Interoperable and Reusable: guiding principles for scientific data management published in 2016.

Dataset citation and DOIs

How to cite datasets and why persistent identifiers such as DOIs give data creators credit and make data findable.

Open government data

Why and how governments publish data openly, the principles behind it, and how citizens, researchers and businesses reuse it.

Collaboration

Version control for data

Version control for data records every change to a dataset, so teams can review, compare, roll back and reproduce results, as with code in Git.

Reproducible analysis

Reproducible analysis means others can re-run your work, with the same data, code and environment, and get the same result.

Notebooks and collaborative analysis

Computational notebooks combine code, results and narrative in one document, making analysis easier to share, review and build on.

Community and contribution models

How open data communities organise contribution, review and maintenance, borrowing practices from open-source software.

Discovery

Data catalogs

A data catalog is a searchable inventory of an organisation’s data, with descriptions, owners, lineage and usage, so people can find and trust data.

Metadata and documentation

Metadata is data about data. Good documentation, from dataset descriptions to data dictionaries and READMEs, is what makes shared data reusable.

Trust

Data lineage

Data lineage traces where data comes from, how it is transformed and where it goes, so teams can debug issues, assess impact and prove provenance.

Data quality

Data quality describes whether data is fit for its intended use, measured across dimensions such as accuracy, completeness, consistency, timeliness and validity.

Data governance

Data governance is the set of roles, policies and processes that decide how data is managed, protected, accessed and used across an organisation.

Data stewardship

Data stewards look after specific datasets: their definitions, quality, documentation and access, acting as the go-to person for users.

Access control for data

Access control decides who can see or change which data. Common models include role-based (RBAC) and attribute-based (ABAC) access, plus row- and column-level security.

Privacy and the GDPR

How data protection law, particularly the EU General Data Protection Regulation (GDPR), shapes the way personal data can be collected, shared and analysed.

De-identification and anonymisation

The difference between pseudonymised and anonymised data, and common techniques such as aggregation, generalisation, k-anonymity and differential privacy.

Data observability

Data observability monitors the health of data and pipelines: freshness, volume, schema changes, distribution and lineage.

Sharing

Data sharing: models and good practice

How organisations share data: open publication, controlled access, data-sharing agreements, live shares and APIs, with the trade-offs of each.

Data clean rooms

A data clean room lets organisations analyse combined data together without exposing each other’s raw records.

Data marketplaces and exchanges

Data marketplaces and exchanges are platforms where organisations publish, discover and obtain datasets, free or paid, under defined terms.

Interoperability

Open standards and formats

Open formats such as CSV, JSON, Parquet and Arrow, and metadata standards such as DCAT and Frictionless Data Package, make data portable and interoperable.

APIs for data access

How APIs (REST, GraphQL, SQL endpoints) provide programmatic, up-to-date access to data, and what makes a data API good to use.

Architecture

Data mesh

Data mesh is a decentralised approach to data architecture built on domain ownership, data as a product, self-serve infrastructure and federated governance.

Data products

A data product is a dataset or data service built and maintained for its users, with an owner, documentation, quality guarantees and a stable interface.

Data contracts

A data contract is an agreement between data producers and consumers that defines schema, meaning, quality and change rules for a dataset.

Warehouse, lake or lakehouse?

The differences between data warehouses, data lakes and lakehouses, and how each supports analysis and collaboration.

ETL vs ELT

ETL transforms data before loading it into storage; ELT loads raw data first and transforms it inside the warehouse or lakehouse.

Analysis

Semantic layers and shared metrics

A semantic layer defines business metrics and dimensions once, so every tool and team calculates “revenue” or “active user” the same way.

Work with ex omnia

Contact details for ex omnia will be published here once confirmed.

Contact Read the guides