Data Practitioner Perspectives: A 2024 Research & Insight Brief
Drawing on insights from professionals averaging 15+ years of industry experience across 48 countries, comprehensive research conducted by the Modern Data Company and Modern Data 101 Community offers an unvarnished look at how data teams spend their time, effort, and resources. The findings, based on extensive survey data collected April through June 2024, challenge many common assumptions about today’s enterprise data operations.
The Hidden Costs of Inefficiency in Data Teams
Data teams are stuck in a resource trap. While executives dream of AI transformation and data-driven decisions, their technical teams are spending most of their time on something else entirely: trying to understand what the business actually needs. Our global survey reveals a stark disconnect between where data teams should be spending their time and where it's actually going.
The numbers tell a compelling story. A surprising 68% of data professionals report that understanding business requirements consumes the majority of their working hours. This misalignment isn't just about initial planning; it creates costly ripple effects throughout project life cycles:
60% of data practitioners report having to rework their tables, dashboards and data products on a regular basis because they are not meeting business needs
Getting initial project approval is consistently rated as the most time-consuming collaboration task. This constant cycle of misalignment and rework drains resources, delays project timelines, and ultimately diminishes the impact of data initiatives across the organization.
"This isn't just about wasted time," one survey respondent noted. "It's about missed opportunities and delayed value creation."
The Maintenance and Integration Treadmill
While teams should be focusing on innovation and value creation, they're trapped in a cycle of maintenance and updates. Our survey reveals that 63% of data practitioners spend more than 15-20% of their time on maintenance work, and 10% spend more than 25% of their time on maintenance–-updating schemas, managing data quality, and modifying pipelines.
Integration efforts are a particular pain point, with 42% of developers reporting that these efforts slow them down and 38% stating that integrations are the costliest part of maintaining their data infrastructure. This relentless focus on upkeep not only drains resources but also throttles the agility and productivity needed to drive meaningful business outcomes.
Moreover, 70% of respondents say they have to use more than 5-7 different tools or work with 3-5 vendors for different tasks around data quality and dashboarding. As a result, about 40% of the respondents agree that they spend more than 30% of their time jumping from one tool to another, ensuring the tools work well together. This not only wastes time, but can drive mistrust as varied tools often have different answers. As one responder said, “It just makes stakeholders lose trust in the data, because they don’t understand why there are three different answers to the same question.”
This maintenance overhead creates a double burden: not only does it consume precious resources, but it also prevents teams from focusing on strategic initiatives that could actually reduce this maintenance burden in the long term.
Access and Discovery: The Waiting Game
In an age where business moves at digital speed, data access shouldn't be a bottleneck. Yet our survey reveals a sobering reality: Business experts are looking to get access to available data immediately, but for more than 70% of respondents, access takes more than a day, and for 20%, more than five days (often when the need is over). What is also notable is that even after getting access, 65% of respondents say they need to spend more than 15-20% of their time figuring out what data should be used to implement any new data project or product.
The Impact on Innovation
The cumulative effect of these challenges creates a clear innovation deficit. When teams are:
- spending most of their time understanding requirements,
- spending days for basic data access,
- and regularly reworking completed projects…
...there's precious little time left for the innovative work that could transform their organizations.
The underlying issue isn’t necessarily the amount of data or ineffective data engineers —it’s the lack of an overarching strategy for data activation. With 62% of data practitioners reporting that getting access to the right data at the right time is the most challenging in terms of effort and time spent, clearly the current “collect everything” mindset isn’t working.
Companies are discovering that storing massive amounts of data "just in case" isn't practical, especially when even data experts struggle to access what they need. A better approach is to establish focused, high-quality datasets aligned with specific business goals. By distinguishing between frequently-used data and archived information, companies can reduce resource and storage costs and drive innovation faster.
Breaking the Cycle
To break the cycle of inefficiency, imagine if data teams managed data like any other business product, more like software, with clear development stages and maintenance plans. This approach focuses on three key areas:
- Building stronger connections between technical and business teams so everyone understands what data is needed and why.
- Giving teams direct, self-service access to business-ready data they can trust and understand.
- Using automation to handle repetitive tasks and maintain data quality.
This product-focused approach can help ensure data actually delivers value to the business.
This article is based on a global survey of 232 data practitioners conducted in April-July 2024, representing organizations across 48 countries. Respondents averaged 15+ years of industry experience.
