It seems that every big player in the data industry today claims to be a real “data platform,” or at least indispensable in building one. Finding a way through this sea of options requires some clarity about what a “modern data platform” is and does.
A modern data platform is a collection of tools and capabilities that, together, allow organizations to become fundamentally “data-driven.” It is based on three core principles and six essential functions.
- Enable self-service for a diverse range of users
- Enable “agile” data management
- Be flexible, set up quickly, and pay-per-use
This article, the first of two, will examine these core design principles to show what users should expect from any modern data platform worthy of the name. The next article in the series will examine the essential functions of a data platform, some current tools to meet those functions, and how each is enhanced by The Modern Data Company’s data operating system, DataOS®.
The 3 Core Principles of a Modern Data Platform
A modern data platform must move and adapt faster than ever to keep up with the diversity of its data and data users. There are three core principles that make a data platform truly “modern.”
Enable self-service for a diverse range of users
In a nutshell, people shouldn’t need an analyst to help them understand their company’s data. According to Facebook Global Data Program Manager Maria Tarasidou, “What happens in big tech companies is that there’s no role that is actually a Data analyst role. Everyone is an analyst .” Because organizations eventually follow precedents set by big tech, everyone should be able to find the data they need quickly and easily. This means that a modern data platform must be intuitively usable by almost everyone in the organization.
Users should be able to discover and understand not only the data but also metadata such as column descriptions and lineage. They should be able to gain insight from their data with minimal involvement by IT.
Enable “agile” data management
Legacy data platforms were complex, opaque, and slow. Everything went through IT, by way of interminable ETL jobs. And don’t even think of making even the smallest change to your query; you’ll go to the back of the queue and start all over again.
Business organizations can no longer tolerate that system. Users (who, again, should be nearly all employees) should have direct access to data, under rules and access privileges set up by IT but transparent to most users.
Compute should be elastic and scalable based on user needs. Ideally, the scaling should happen automatically according to usage patterns identified by IT. Users should not have to worry about whether they will have the compute they need, but if they do need to ask for more, then the process should be quick and easy. The simplest way to achieve this is through cloud-based compute.
Be flexible, set up quickly, and pay-per-use
Using cloud-based compute and storage makes a modern data platform flexible to meet users’ changing needs. They are fast and easy to set up, without complex installations or pilot projects that last months. Set up the client, connect to the servers, and go.
Pricing structures should be flexible as well, based on pay-per-use rather than monthly or annual lump sums. There should be nothing paid up-front, no massive licensing fees that force businesses to deal with sunk costs.
Flexibility also means the ability to meet changing needs over the long term. As technology evolves, new solutions should be based on open standards and APIs that allow them to be added to the stack easily. Tools should never lock users in or lock other tools out.
Each of these principles (self-service; fast, transparent access to data and computing resources; financial and technical flexibility) are aimed at one goal: making a data platform that users simply use without worrying about how it all works or whether it can grow as their needs change.
DataOS checks every one of these boxes. It is a data operating system designed for transparent, intuitive access to the tools that make up the modern data platform. Our next article will explore these tools and show how DataOS improves each of the six essential functions of any data stack.
For more information on how Modern integrates these core principles into our data operating system, DataOS, download our Modern DataOS Overview.
