The Value of Data

We treat a data set the same way that an eCommerce business treats products and services. Each data set has a description, schema definition, profile of the data, quality index, tags, other datasets that are similar, top users and top queries on the data set, etc. By productizing data and attaching the semantics to the data, we can then use our technology to move this data to any user or system in the format that they need.

The key to extracting value from data lies in the ability to understand the quality of the data and ways to turn data into high-quality information. We do this by profiling all datasets that are ingested and providing detailed analysis on quality attributes like cardinality, completeness, missing values, uniqueness, etc. We also run business-specific data validation rules to help objectively quantify data quality.

Access to data, application data silos and fragmented data across the enterprise are some of the key reasons why data usability is as low as it is today. The first step in remedying this is being able to catalog all things data (data sets, jobs, metrics and KPIs) and making them available with a Google-like search interface. This kind of access to data and the jobs that manipulate the data provide instant access and understanding of the data and empowers users to data-surf vs the data-drilldowns that they are using today.

The ability to support ingesting data in ANY format (from mainframes to streaming systems to IoT/5G, structured, unstructured to semi-structured, batch to real time to one-time loads), and being able to make that data available in a secure and compliant manner to be used in ANY format on the other side is the key to providing plug and play integrations and moving away from the ETL way of managing data. DataOS makes data truly interoperable.

Making data available for various uses like analytics, data science, automation, etc. in the format and security classification relevant for that particular use is one of the core aspects of a Data Fabric architecture and is a foundational capability of DataOS. This provides an ability for business teams to use the same data for various needs without the need for use case specific data pipelines, transformations, data copies, etc.

Data ownership is a construct that has hindered data usability in many organizations. This approach to data governance leads to many data prisons across the enterprise which are tough to govern and manage. This also creates high risk of data breaches. We shifted the data ownership paradigm to a data access paradigm. In this new approach, data is owned by the enterprise and not individual teams. Any user or system that needs access to the data can do so in a compliant manner by leveraging foundational capabilities such as row and column level redactions, data abstraction, data masking, etc. These capabilities, along with data observability, ensures that every user or system in the company has access to the data that they need, in the format that they need and with the right governance applied to it.

We are a post GDPR company. As such, we don’t have the baggage of rigid architectures or old technologies that limit how large systems can be compliant with standards such as GDPR, CCPA, etc. We use tagging extensively to manage security and priority classifications at the most atomic level. This makes new regulatory compliance needs a business logic solve rather than a re-architecture solve that is often prohibitively expensive and takes months or years. These approaches allow us to innovate on how we manage governance. Our tag-based governance engine provides customers with the flexibility to provide role based, attribute based and tag-based access controls. This empowers teams to set up conditional access controls like the ability to access a data set on a certain network, or during a certain period of time or other such rules.