Incident sheet · /data

Filed
Read
4 min · 792 words
Severity
Routine

Conversational Analytics Shifts Data-Quality Burden to Business Users

Analysts at Accenture, IDC and Juniper Research warn conversational analytics hides definitions, join errors and permission gaps until after users act on AI-generated answers.

AI analytics raises the stakes for showing your work
FeedAI analytics raises the stakes for showing your work — AI-generated

Event log

  1. Conversational analytics shifts error detection from dashboard review to post-decision discovery.
  2. Accenture's Teresa Tung: wrong term interpretation can invalidate everything that follows in an AI answer.
  3. IDC's Megha Kumar: a bad data join can double-count orders and inflate revenue without looking implausible.
  4. LatentView CEO Sonal Ramrakhhani: conversational-tool access to sensitive data must match established tools.
  5. Investigators should log question, requester, generated query, model version, permissions and output.

Conversational analytics moves the point of failure from dashboard review to post-decision discovery, and enterprise data teams are not ready. Vendors and analysts this week detailed how chat-based querying of business data hides definitional choices, join errors and permission gaps that established dashboards caught before publication.

The core problem is structural. A question about quarterly sales can return a convincing number while leaving invisible which transactions were counted, how revenue was defined, and whether the employee was entitled to see all the information used. Errors that data teams once caught while building and reviewing a dashboard may now surface only after someone acts on an answer.

Connecting conversational analysis to automated workflows raises the stakes further. An incorrect calculation or an overlooked access restriction can influence a business process before an analyst reviews the result.

Why can two correct answers disagree?

Teresa Tung, global advanced data lead at Accenture, said conversational tools must first interpret terms that carry different meanings across departments.

"If it gets that interpretation wrong, everything that follows can be wrong too," she said.

Sales and finance can ask for last quarter's customer count and receive different answers without either number being miscalculated. Sales may count everyone who made a purchase. Finance may count only accounts with active contracts, Tung explained.

Unless the tool makes its interpretation explicit, employees can mistake a disagreement over definitions for a discrepancy in business performance. A published dashboard allowed data teams to resolve those differences before the figures reached users. Conversational tools can leave that interpretive choice to the AI on every query.

What errors stay invisible in the output?

Megha Kumar, research vice president for analytics and AI at IDC, said errors in how datasets are combined can distort results without making them look implausible.

"A bad join can count orders twice and inflate revenue or drop records that don't match and leave out part of the business," she said.

Kumar also cautioned that a tool might use information collected for another purpose, or filter results using a sensitive attribute. Checking whether the final number seems reasonable would not necessarily reveal either problem.

Users unfamiliar with the underlying data are particularly poorly placed to catch these failures, according to Nick Maynard, vice president of research at Juniper Research.

"Without full familiarity on the data sources, an answer that meets a user's preconceptions can be accepted at face value," he said.

How must permissions change?

Conversational access can expose inconsistencies between systems that previously served different audiences. An employee permitted to view purchasing trends should not gain access to individual customer records simply because a tool draws on both sources.

Sonal Ramrakhiani, CEO of LatentView Analytics, said organizations must preserve existing restrictions as they introduce conversational interfaces.

"Companies need to make sure employees' access to sensitive information through a conversational tool is identical to the more well-established tools in the business," she said.

Tung noted that conversational analytics does not inherently bypass controls. The risk rises when identity and permissions are enforced inconsistently across the systems supplying an answer. Data managers working with security teams therefore need to examine access throughout the analysis, including the sources consulted and the information returned.

What evidence should teams retain?

Before an analysis informs a consequential decision, users need to know how their question was interpreted, which definitions applied, and how current the data was. Tung said the evidence trail should also allow teams to reconstruct sources, transformations and lineage (the record of data's origin and movement through systems).

For automated actions, Kumar recommended rules governing which sources are acceptable, plus records connecting each action to the answer that triggered it. Investigators need the original question, the requester, the generated query, the model version, the applicable permissions and the resulting output.

Those records serve two distinct responsibilities. Data and analytics teams must investigate the analysis itself. Business owners must review decisions affected by a mistake, Ramrakhiani said. Retaining only the final answer leaves both groups without enough evidence to establish what happened.

"That requires keeping a history of the analysis so the team can reproduce the result, even if the underlying data has since changed," she said.

What should data leaders do now?

The guidance from Accenture, IDC, Juniper Research and LatentView Analytics converges on four controls:

  • Make the tool's interpretation of business terms visible in every answer.
  • Audit join logic and source usage before results feed automated workflows.
  • Enforce permissions consistently across every system a query touches.
  • Log the full question-to-output chain, including model version and generated query.

Data managers remain accountable for these choices as business teams adopt the tools. The dashboard era embedded them in review processes. The conversational era leaves them to governance.

via techtarget.com (Source)

Filed under

  • conversational-analytics
  • data-governance
  • data-quality
  • access-control
  • data-lineage
Share this article:

More from Rebecca Stone

Rebecca Stone

Show full bio

Senior reporter covering money, funding and market shifts around enterprise information technology and the leaders who run at CIO Monitor.

199 articles

The newsroom digest

One email a day with the stories that matter. No noise.

By subscribing you agree to the privacy policy.

Cross-references

‹ Previous articleNext article ›