An AI data agent can find a number. You still need to know what it counts.

OpenAI says its Data agent can use an organization’s business terms, metric definitions, calculations and data relationships, drawing on approved sources such as semantic layers, dashboards and connected data platforms. Administrators choose the data connections and roles available to it, and queries follow the connected account’s existing table, row and column permissions. Those are product-design claims from the launch, not an independent test of every company’s setup.

The important constraint is less glamorous: the agent can only inherit the definitions and access patterns a company has already made coherent. If two teams mean different things by “active customer,” the assistant cannot make that disagreement disappear by drawing a clean line chart. It may make the disagreement arrive sooner, which is useful only if the result points to the definition it used.

NIST’s AI Risk Management Framework is voluntary guidance rather than a product checklist, but its basic direction fits here: manage risk in the actual context of use. A number used to decide staffing, renewals or spending deserves more than a confident sentence.

For a data answer that will change a decision, leave a small evidence strip beside the conclusion:

- the exact metric definition - the date range and comparison period - the source tables, reports or documents used - the population included and excluded - late, missing or manually corrected data - the last refresh time

This does not need to become a technical report. It can be one expandable line under a chart. The point is to stop a summary from becoming more authoritative than the record beneath it.

Try the same standard on a practical question: “Why did renewals slow down?” A useful answer separates a vendor’s claimed driver from an observed change in the data, says how many renewals it examined, names the period, and marks the next check as a check—not as a discovered fact.

If your team is evaluating an AI data assistant, do not start with a broad request for an executive dashboard. Pick one recurring question that already costs somebody time: which accounts need a follow-up this week, why did support volume change, or where did spend move from last month to this one?

Have the agent produce an answer, then ask a person who knows the work to trace three claims back to the underlying records. Record how long that takes, what needed correction, which caveats were missing, and whether the result changed the next action. Compare that with the usual report.

The test is not whether the chart looks right in a meeting. It is whether a person can make a decision without quietly inheriting a bad definition, a stale extract or somebody else’s unexplained assumption.