The Value of Data

We treat datasets the same way that eCommerce businesses treat products and services. Each dataset has a description, schema definition, profile of the data, quality index, tags, similar datasets, top users, top queries on the dataset, and more. By productizing data and attaching the semantics, we can then use our technology to move this data to any user or system in the format that’s needed.

The key to extracting value from data lies in the ability to understand the quality of the data and the ways to turn data into high-quality information. We do this by profiling all datasets that are ingested and providing detailed analysis on quality attributes like cardinality, completeness, missing values, uniqueness, etc. We also run business-specific data validation rules to help objectively quantify data quality.

Access to data, application data silos, and fragmented data across the enterprise are some of the key reasons why data usability is as low as it is today. The first step in remedying this is being able to catalog all things data (i.e., data sets, jobs, metrics, and KPIs) and making them available with a Google-like search interface. This kind of access to data and the jobs that manipulate the data provide instant access and understanding of the data and empowers users to data-surf vs the data drill-downs that they are using today.

The ability to support data ingestion in ANY format (e.g., from mainframes, streaming systems, and IoT/5G to structured, unstructured, or semi-structured to batch, real-time, or one-time loads) and being able to make that data available in a secure and compliant manner to be used in ANY format on the other side is the key to providing plug and play integrations and moving away from the ETL way of managing data. DataOS makes data truly interoperable.

Making data available for various uses like analytics, data science, and automation, etc. in the format and security classification relevant for that particular use is one of the core aspects of a data fabric architecture and is a foundational capability of DataOS. This enables business teams to use the same data for various needs without case-specific data pipelines, transformations, and data copies.

Data ownership is a construct that has hindered data usability in many organizations. This approach to data governance leads to many data prisons across the enterprise which are tough to govern and manage. This also increases the risk of data breaches. We shifted the data ownership paradigm to a data access paradigm. In this new approach, data is owned by the enterprise and not individual teams. Any user or system that needs access to the data can do so in a compliant manner by leveraging foundational capabilities such as row and column level redactions, data abstraction, and data masking. These capabilities, along with data observability, ensures that every user or system in the company has access to the data they need, in the format they need, and with the right governance applied to it.

We are a post-GDPR company. As such, we don’t have the baggage of rigid architectures or old technologies that limit how large systems can be compliant with standards such as GDPR, CCPA, etc. We use tagging extensively to manage security and priority classifications at the most atomic level. This makes new regulatory compliance needs a business logic solve rather than a re-architecture solve that is often prohibitively expensive and takes months or years. These approaches allow us to innovate on how we manage governance. Our tag-based governance engine provides customers the flexibility to control access based on roles, attributes, and tags. This empowers teams to set up conditional access controls like the ability to access a dataset on a certain network or during a certain period of time.