Why DQ Programmes Fail and How DMAIC Fixes It

The Scale of the Problem

The cost of poor data quality in financial services is not a minor inefficiency it is a material operational, regulatory and commercial risk.

Gartner's Data Quality Market Survey estimates the average organisation loses $12.9 million annually to poor data quality. In financial services, where a single data defect can cascade into a failed settlement, a regulatory breach or an incorrect client report, the real figure is considerably higher. The consequences are also more visible and more consequential than in most industries.

A 2025 IBM Institute for Business Value report found that 43% of chief operations officers now identify data quality as their most significant data priority. In the Bank of England's 2024 survey of AI in UK financial services, four of the top five perceived current risks were data-related. In asset servicing and custody alone, data defects translate directly into failed settlements and CSDR penalties, corporate action errors leading to client claims, NAV inaccuracies and reconciliation breaks. In asset management, bad data drives investment decision risk, IBOR exceptions and ESG data fragmentation. In wealth management, the consequences show up as client reporting failures, suitability assessment gaps and Consumer Duty compliance exposure.

The question is not whether the problem is real. It clearly is. The question is why, after years of investment, it keeps persisting.

Why Traditional DQ Programmes Fail

There are five structural reasons that traditional data quality programmes in financial services fail and they are remarkably consistent across every type of institution.

1. Rule explosion and false positive fatigue

Thousands of static rules are built with no business context. Over time they multiply. Rules lack lineage and impact context, a missing field in a deprecated staging table triggers the same severity alert as a missing field in a financial compliance report. Noise replaces signal. Operations teams learn to ignore the alerts. The programme loses credibility. Real issues get buried in a queue nobody trusts.

2. Batch-oriented, retrospective controls

DQ checks run overnight. By the time a defect is detected it has already propagated to trade instructions, client reports and regulatory submissions. Many organisations only evaluate data after it lands in the warehouse by then corrupted records have already been ingested, processed and consumed by live systems. You are always catching up, never preventing.

3. Disconnected from business impact

Controls are built by engineers without reference to the operational processes that actually matter. The root cause of a failed settlement or an incorrect NAV is never traced back to the originating data defect. Fixing the symptom becomes the daily ritual. The cause is never addressed.

4. Siloed ownership

DQ is owned by Technology, not the business functions that feel the pain every day. Issues are logged but stewardship is absent. Without designated data owners and active stewards accountable for specific domains, the same defects recur indefinitely regardless of what tooling is in place or how much has been spent on it.

5. No measurable value realisation

There are no clear ROI metrics connecting data quality investment to operational outcomes. Poor data quality often goes unnoticed because its impact rarely appears at the point of failure instead it surfaces downstream as lost revenue, inefficiencies and compliance risks. When that connection to business impact is never made explicit, DQ investment cannot be defended to boards. Programmes stall when budgets tighten. The cycle repeats.

The result is a DQ maturity curve where most institutions remain stuck at the reactive or rules-based stage detecting problems after the fact rather than preventing them at source. More technology gets layered on top of a structurally broken approach. The problem does not get smaller. It gets more expensive.

A Different Approach : AI-Enabled DMAIC

The firms that are making real progress on data quality have one thing in common. They stopped treating it as a technology problem and started treating it as a business programme with process, ownership, governance and accountability at the centre, and technology as the enabler rather than the solution.

At Keyrus, we have reimagined the DMAIC framework a proven quality management lifecycle specifically for AI-native financial services environments. The real differentiator is not the AI. It is the operating model that surrounds it: the people, the process, the governance structure and the policy framework that connects data quality controls to the business outcomes that actually matter. AI accelerates every stage of the framework. But without the structural foundation, it accelerates the wrong things.

Here is what the framework looks like in practice:

Define : establish what good looks like and who owns it

Most DQ programmes start by cataloguing data assets and writing rules. Ours starts with the business. We work with operations, risk, compliance and finance leaders to identify the critical data elements driving the highest operational and regulatory risk typically the 5% of data driving 80% of failures. We define data ownership explicitly, with named accountable owners in the business, not the data team. We establish what good looks like in business terms not as a technical threshold, but as an operational standard tied to specific processes and outcomes. AI accelerates this by profiling data at scale and mapping relationships across systems, but the definitions come from the business.

Measure : controls that reflect business reality, not technical coverage

Traditional DQ programmes measure everything equally. A missing field in a deprecated staging table carries the same weight as a missing field in a regulatory submission. This is why alert volumes become unmanageable. Our approach ranks every control by its potential business impact before it is deployed failed settlements, NAV errors, client reporting failures, regulatory breaks. Automated rule generation reduces the manual burden on data teams, but every rule has a business owner who has validated its relevance. Event-stream monitoring replaces overnight batch cycles so controls operate continuously rather than retrospectively.

Analyse : root cause, not symptom management

This is the stage most DQ programmes skip entirely. Issues are detected, logged and fixed and then recur the following month because nobody traced them back to their origin. Our root cause analysis connects downstream failures to their upstream cause across systems and processes. When a settlement fails, we do not just fix the record we identify whether the defect originated in a data entry process, a system integration, a vendor feed or a governance gap, and we address the source. AI accelerates the diagnostic process significantly, reducing root cause identification from days to hours. But the fix is almost always a process or ownership change, not a technical one.

Improve : remediation that is owned by the business and calibrated to risk

Remediation authority is one of the most overlooked dimensions of DQ programme design. Who is allowed to correct what, under what circumstances, with what governance? We define a clear remediation policy that distinguishes between low-risk automated corrections ISIN enrichment, duplicate detection, format standardisation and high-impact decisions that require human review, such as NAV adjustments, regulatory submissions and AML/KYC changes. AI agents handle the high-volume, low-risk corrections autonomously. Everything else goes through a defined human-in-the-loop process with an audit trail. The policy governs the technology, not the other way around.

Control : embedded standards, not periodic audits

Sustainable data quality requires controls that are embedded into operational workflows permanently not activated before a regulatory deadline and deprioritised afterwards. We design DQ standards into the processes and systems that create data, not just the ones that consume it. This shift-left approach means defects are prevented at source rather than caught late. Data stewards drawn from business functions, not the data team own the ongoing monitoring of their domains and escalate issues through a defined governance process. Continuous monitoring agents provide real-time visibility, but the accountability sits with the business.

The result is a DQ programme that does not depend on a central data team to maintain it, does not collapse when the project team moves on, and does not generate thousands of alerts that nobody acts on. It becomes an operational standard owned by the business, enforced by process, accelerated by AI.

What This Looks Like in Practice

We have applied this framework inside one of the world's most complex global custody and asset servicing environments an institution operating across custody, fund administration, securities lending and corporate actions at scale across dozens of markets.

The outcomes were measurable and specific:

• 40–60% reduction in manual remediation headcount cost

• 70%+ reduction in false positive alert volumes

• 3–5x faster root cause identification

• 80%+ of low-risk remediation tasks handled autonomously

Beyond the numbers, the more significant shift was structural. Data quality moved from a cost centre to an embedded operational standard. Business teams owned the problem. Controls were designed around the outcomes that mattered settlement efficiency, regulatory readiness, client reporting accuracy not around technical coverage metrics that nobody in the business could interpret or act on.