real world example of an open source data catalog

The challenge

Organizations with a growing data landscape face several challenges as they implement a holistic data governance strategy that allows people to securely identify and use relevant data sets while having a clear and quick process to approve and provide access to this data. The implementation of a data catalog should enable employees of our customer to better collaborate and easily track KPIs and was one part of the developed strategy. To make this possible, a data stewardship model had to be established. The goal of the project was to find out which data catalog solution (commercial or open-source) is the best choice for specific use cases.

Our solution

In order to evalutate how well the Datahub covers or can be adapted to the customers’ use cases, the data catalog was implemented in a flexible environment hosted by Contiamo. Our team adapted the metadata model of the Datahub according to the requirements, built a multi-layer authorization management, developed individual metadata assets and implemented a clear view of the business and technical lineages.

Tech stack

LinkedIn Acryl Datahub

Kubernetes

OTC

The project

The project was part of the development of a comprehensive data governance strategy. The customer internally assessed goals and potential pilot projects along different departments. Based on these defined requirements, they compared various commercial and open source data catalogs. The most promising two catalogs were tested in 2-month proof of concepts.

We kicked off the proof of concept by leading a workshop to prioritize the most impactful functionalities within the chosen use case. These include:

Comprehensive customization of the business glossary
Creation of a technical & business data lineage
Setup of a flexible, powerful authorization management system
Customization of the data catalogs metadata model
Demonstration of collaboration capabilities within the tool and across other tools

Contiamo then quickly implemented and provided these solutions using a SaaS model within our own secure cloud environment. Each implementation was iteratively evaluated and improved with the project team. We also took care of data protection, user onboarding and deep-dive workshops. In the end, business users intensively tested the two data catalog and gave very positive feedback on the open source solution.

‍

Achieved benefits

Introduction of the team and IT to a promising open source technology

Creation and implementation of an individual stewardship model

Open source tools

The goal of this proof of concept was to evaluate an open source data catalog for the customer's individual use cases. The Contiamo team has successfully demonstrated that the LinkedIn Datahub offers great flexibility and a high degree of customization. The employees were very satisfied with the usability of the catalog. Sharing various best practices and approaches for the implementation of holistic data governance also supported the customer in its upcoming challenges.

← BACK TO ALL Case studies

Enabling data democratization through modern open source technology

The challenge

Our solution

Tech stack

The project

Achieved benefits

Curious to learn how we can help your business?

Unlock the value of your data.