Quickly and easily analyze data sets

The company's challenge was to design a data ecosystem that would provide experts with rapid analytics and deep insights.

dashboard data analysis

Enable efficient data analysis in 4 steps

1. Data exploration
2. Data
analysis
3. Data
curation
4. Data
sharing

1. Make all data explorable

A matching data catalog allows data experts to search all available assets and data structures in seconds. Automation was the key to success here. This means that the data always remains up to date without enormous manual effort. Analysts can also use the catalog to find out at any time whether the data is up-to-date and suitable for the question at hand.

In addition to clear documentation and business context, a powerful search helps to find the data needed efficiently. The search function was a core criterion when selecting a suitable catalog tool. Diverse filtering and sorting functions were important for the user group, as the available data base had a very large volume. Basically, a catalog with its features should always fit the real user group.

discover all data in a data catalog like linkedin datahub
Browse data
directly analyse and understand data sets using data profiling
Quick insights

2. Analyze data

Data profiling tools and functions allow data experts to quickly gain a good first impression of the identified data. Statistics, real data distributions and properties of the data set are useful information, as are sample values.

Depending on the issue and the team, the direct connection of the data catalog with the BI tool is very helpful. Many companies rely on Tableau, PowerBI or Redash to provide deeper insights. A central overview was also created to provide the data teams with a centralized and clear view of all other tools used for analysis. The goal was to make the jump from the data directly to the tool of choice as uncomplicated and fast as possible.

3. Curate data

After data for a use case was successfully and quickly identified and analyzed, typical steps of data preparation followed: cleansing, enrichment and combination with other data.

An optimal way to further process the given data is offered by SQL. The modern data architecture implemented in this project allows flexible querying of data from different sources.

Overall, it was of central importance for efficient data curation to provide all employees with the most suitable tools. To ensure this, in addition to the development of several interfaces, the data catalog has been established as a central documentation location. In this way, flexibility and transparency can be combined in real operations.

curate datasets using modern tools like dbt
Efficiently transform data
export data assets directly into connected tools via API and make data available
Make data assets available

4. Share data

Once data was successfully found, inspected, and curated via SQL, it was critical to share these new datasets with colleagues and the departments. The same of course applies to insights gained in any other form, e.g. reports, notebooks and models.

The created queries can be stored in the desired connected data source (e.g. the cloud data warehouse). Via the catalog, they are thus directly available to any authorized user. In this way, BI departments, for example, can be centrally supplied with prepared data records for evaluations without creating data silos.

quotes
The key to success was to create a networked architecture. This involves creating global transparency across the multiple components of the structure. It is important to retain and improve the flexibility of both the teams and the tools. This is how agile working with data becomes possible in the long term.
quotes
Mike Linthe, COO
Let's talk together about your challenges.

We are always there for our customers. Simply contact us for a non-binding conversation.

Mike Linthe
Kira Lenz
Michael Franzkowiak
Ian Solliec
Contact