tmdc logo white
tmdc logo dark

The Core Principles of a Modern Data Platform

by

The Core Principles of a Modern Data Platform

by Apr 13, 2022Data Management

It seems that every big player in the data industry today claims to be a real “data platform,” or at least indispensable in building one. Finding a way through this sea of options requires some clarity about what a “modern data platform” is and does.

A modern data platform is a collection of tools and capabilities that, together, allow organizations to become fundamentally “data-driven.” It is based on three core principles and six essential functions.

  • Enable self-service for a diverse range of users
  • Enable “agile” data management
  • Be flexible, set up quickly, and pay-per-use

This article, the first of two, will examine these core design principles to show what users should expect from any modern data platform worthy of the name. The next article in the series will examine the essential functions of a data platform, some current tools to meet those functions, and how each is enhanced by The Modern Data Company’s data operating system, DataOS®.

 

The 3 Core Principles of a Modern Data Platform

A modern data platform must move and adapt faster than ever to keep up with the diversity of its data and data users. There are three core principles that make a data platform truly “modern.”

 

Enable self-service for a diverse range of users

In a nutshell, people shouldn’t need an analyst to help them understand their company’s data. According to Facebook Global Data Program Manager Maria Tarasidou, “What happens in big tech companies is that there’s no role that is actually a Data analyst role. Everyone is an analyst .” Because organizations eventually follow precedents set by big tech, everyone should be able to find the data they need quickly and easily. This means that a modern data platform must be intuitively usable by almost everyone in the organization.

Users should be able to discover and understand not only the data but also metadata such as column descriptions and lineage. They should be able to gain insight from their data with minimal involvement by IT.

 

Enable “agile” data management

Legacy data platforms were complex, opaque, and slow. Everything went through IT, by way of interminable ETL jobs. And don’t even think of making even the smallest change to your query; you’ll go to the back of the queue and start all over again.

Business organizations can no longer tolerate that system. Users (who, again, should be nearly all employees) should have direct access to data, under rules and access privileges set up by IT but transparent to most users.

Compute should be elastic and scalable based on user needs. Ideally, the scaling should happen automatically according to usage patterns identified by IT. Users should not have to worry about whether they will have the compute they need, but if they do need to ask for more, then the process should be quick and easy. The simplest way to achieve this is through cloud-based compute.

 

Be flexible, set up quickly, and pay-per-use

Using cloud-based compute and storage makes a modern data platform flexible to meet users’ changing needs. They are fast and easy to set up, without complex installations or pilot projects that last months. Set up the client, connect to the servers, and go.

Pricing structures should be flexible as well, based on pay-per-use rather than monthly or annual lump sums. There should be nothing paid up-front, no massive licensing fees that force businesses to deal with sunk costs.

Flexibility also means the ability to meet changing needs over the long term. As technology evolves, new solutions should be based on open standards and APIs that allow them to be added to the stack easily. Tools should never lock users in or lock other tools out.

Each of these principles (self-service; fast, transparent access to data and computing resources; financial and technical flexibility) are aimed at one goal: making a data platform that users simply use without worrying about how it all works or whether it can grow as their needs change.

DataOS checks every one of these boxes. It is a data operating system designed for transparent, intuitive access to the tools that make up the modern data platform. Our next article will explore these tools and show how DataOS improves each of the six essential functions of any data stack.

For more information on how Modern integrates these core principles into our data operating system, DataOS, download our Modern DataOS Overview.

Subscribe to
Our Blog

Be the first to know about the latest insights from Modern.


People Also Read

Latest Resources

The Modern Data Company Brief

The Modern Data Company Brief

The Modern Data Company BriefThe Modern Data Company is radically simplifying data architecture with its paradigm-shifting data operating system, DataOS. We're replacing overwhelm with composability, reinventing governance, and connecting legacy systems to your newest...

DataOS – The Data Product Platform

DataOS – The Data Product Platform

DataOS® – The Data Product PlatformDataOS is the The Data Product Platform pioneered to enable data teams to create, deploy, and manage self-sufficient enterprise-grade data products. These data products are reusable, composable, and compatible across any data stack,...

DataOS and Snowflake – Better Together

DataOS and Snowflake – Better Together

Not Getting Value from Your Data Transformation? Fix itImplementing customer lifetime value as a mission-critical KPI has many challenges. Companies need consistent, high-quality data and a straightforward way to measure CLV. In the past, organizations have struggled...

DataOS® Solution: AI/ML360

DataOS® Solution: AI/ML360

DataOS® Solution:AI/ML 70% of AI initiatives fail and teams spend the vast majority of their time simply prepping data for platforms, leaving very little left over for gaining insights and driving business value. But an AI/ML platform powered by DataOS can achieve...