When AI sounds certain, ask why
AI has become remarkably good at producing answers. But smart business leaders don’t make decisions based on answers alone. They ask where the information came from, what assumptions shaped the conclusion, and how much confidence they should place in the recommendation.
Founder and CEO of Stravito.
Those questions are becoming increasingly important as AI takes on a larger role in enterprise decision-making. Marketing teams are now using it to evaluate campaign concepts. Insights teams are asking it to synthesize years of consumer research. Executives are relying on it to identify growth opportunities, assess competitive threats, and pressure test major investments.
Once AI starts influencing decisions instead of simply accelerating work, understanding how it reached a conclusion becomes just as important as the conclusion itself.
Latest Videos FromTechRadar
Every recommendation deserves an explanation
Consider a CPG firm looking to enter convenience stores while continuing to sell products in supermarkets. The decision calls for balancing dozens of variables, from the impact on supermarket sales and pricing to customer demographics, channel growth, and long-term brand implications.
No single report has all this information. A leader needs to compile it from multiple sources and analyze it comprehensively before deciding whether to pursue the expansion.
AI can dramatically accelerate that process by synthesizing years of research, identifying patterns across hundreds of documents, and surfacing insights in minutes – helping teams to spend less time gathering information and more time evaluating it.
But AI doesn’t eliminate the need for judgment. Leaders are still responsible for understanding the reasoning behind the recommendations they ultimately act on.
The answer tells only part of the story
That’s where some of the most commonly used AI tools today can fall short.
Many AI tools create answers that appear compelling; however, they often contain no indication of how the system developed them.
These answers combine proprietary research, web data, and AI-generated content, with little indication of how each component was utilized and weighed in the final recommendation.
For this reason, many current AI tools operate like black boxes – offering recommendations without the requisite context. This opaque approach may be acceptable for exploratory or non-critical applications. However, decisions involving major investments, new products, or strategic planning require visibility and transparency.
Imagine AI recommends expanding into convenience stores because consumer demand is expected to grow. The recommendation itself may be reasonable, but decision-makers should also understand the sources, which sources carried the most weight, which conclusions are supported by evidence, which rely on inference, and where the available information leaves room for uncertainty.
Without that visibility, it’s difficult to know whether you’re acting on well-supported evidence or simply accepting a convincing narrative.
Uncertainty is fundamental to decision-making
One of the biggest misconceptions about AI is that uncertainty is a weakness. Any degree of uncertainty or equivocation expressed by AI is deemed a bug, not a feature. In reality, uncertainty has always been part of good decision-making.
Experienced leaders don’t expect perfect information. They expect to understand where evidence is strong, where it’s limited, and which assumptions deserve further discussion.
Traditional research naturally encouraged those conversations. However, AI can compress that process into a polished answer, making it easier to overlook what stays uncertain.
Yet those unknowns are often the most valuable output. Recognizing weak evidence, conflicting findings, or missing information gives organizations the opportunity to ask better questions, gather additional research, and avoid making important decisions with a false sense of certainty.
The Glass Box AI model
These principles point toward what I think of as a “Glass Box” approach to AI. Instead of treating transparency as a singular feature, this approach provides greater visibility into the information, reasoning, and uncertainty within enterprise AI.
At its core, every AI output should provide an explanation for its reasoning that is understandable and retrievable. Leaders should be able to examine the evidence evaluated, the filters used, and how the evidence became a conclusion.
Each claim should also include references to exact pages and passages in source materials rather than referring broadly to an entire library of documents requiring manual review. Glass Box AI clearly separates what the source material stated from what was inferred by the AI. Thin evidence should be identified as such and not masked by presentation techniques.
A Glass Box AI approach should also identify gaps in knowledge as well as what was found. Lack of evidence regarding a key assumption should be included in the report so users can consider it during the decision-making process.
Critically, it should preserve user intervention. Leaders should have the ability to question, reject, or adapt an AI system’s conclusions – and record the basis for their rationale. If an AI suggests that convenience stores will allow a CPG firm to charge higher prices, yet a member of the product team believes otherwise, that disagreement should be reflected in the documentation.
Transparency should exist throughout an AI system’s processing cycle, not only once the answer is completed. While working, the system should demonstrate what it is searching for, what it is weighing, and where it is moving toward convergence. This allows users to catch issues early and adjust the weighting before the recommendation is completed.
Trusted AI is Glass Box AI
Organizations increasingly rely on AI to make strategic decisions about enterprise development, capital deployment, product innovation, marketing strategy, and more. In this new reality, “trust me” cannot be an acceptable citation when making these types of decisions.
Enterprises require evidence that can be traced, reasoning that can be challenged, and conclusions that can withstand scrutiny. This is the foundation of Glass Box AI, and what I believe should be built towards, to meet the new enterprise AI standard that must be met.
Because the value of AI will ultimately be measured not by how confidently it answers, but by how confidently organizations can act on those answers.
We’ve featured the best AI website builder.
This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit