In the world of artificial intelligence development, one of the most complex dilemmas is how to make a model honest without sacrificing its usefulness. That is, when an AI agent is unsure about an answer, should it admit uncertainty or fabricate something plausible? The temptation to generate complete but incorrect answers is high, especially when reinforcement learning from human feedback (RLHF) rewards confidence and penalizes ambiguity. We recently conducted an experiment to measure how a simple 'psychological safety prompt' can change the honesty of a language model, and the results offer valuable lessons for any company developing custom software based on AI.
The underlying problem is not new: large language models are trained to maximize the probability of tokens that appear coherent, not necessarily truthful. In production environments, this can translate into costly hallucinations, from erroneous medical diagnoses to inaccurate financial recommendations. The typical solution has been to add behavioral instructions like 'you must say I don't know when you are unsure,' but those rules compete with other directives and the model ends up prioritizing completeness over accuracy. Inspired by Google's Project Aristotle research on psychological safety, we designed a prompt that does not command a behavior, but rather establishes a trust relationship: 'you will not be punished for not knowing.'
The experiment used 40 questions divided into two groups: 20 the model could answer with certainty (programming topics, protocols, databases) and 20 that were impossible to know (future stock quotes, private user data, 2049 events). Each question was run in two conditions: with a neutral base prompt and with the same base prompt plus the safety prompt. We measured both the textual response and the log-probability of the first token to determine whether the model preferred to answer or admit ignorance. The triple hypothesis was: accuracy on known questions should not decrease, uncertainty admission on boundary questions should increase, and the internal confidence (logprob) in the 'cannot answer' option should grow.
The results confirmed the first hypothesis without surprises: accuracy remained at 98-99%, with a slight improvement in one case. That is, the safety prompt did not make the model more foolish. Regarding the second hypothesis, the rate of uncertainty admission rose from 90% to 97%, but with an important caveat: 15 of the 20 boundary questions were already at ceiling (the model already said 'I don't know' even without the prompt). Only 5 questions had room for improvement, and in 3 of them the improvement was significant. The third hypothesis revealed a paradox: at the aggregate level, the prompt reduced the logprob preference for the 'cannot answer' option by -0.72, which could be interpreted as a loss of confidence. However, when disaggregated by question, it was discovered that in the 5 questions where behavioral change actually occurred, the correlation between improvement and logprob increase was +0.949, almost perfect. That is, when the prompt worked, it did so with greater internal conviction, not less. The global average was misleading due to noise from questions already at ceiling.
This finding has direct implications for the design of AI agents in business environments. Many companies deploy models with aggressive quality configurations: execution gates, 'default to execute' directives, self-regeneration pressure. In those contexts, the tendency to fabricate plausible answers skyrockets. A safety prompt, understood not as another rule but as a relational signal, can act as a safety net that protects against failure modes. However, the experiment also showed that in pure API calls, the model is already surprisingly honest (90% admission on boundary questions). The real risk of fabrication is not in the base prompt, but in system pressure.
From the perspective of a software development company like Q2BSTUDIO, these results reinforce the importance of multi-layer verification. It is not enough to trust the direct output of the model; an architecture is needed that validates, corrects, and above all, allows the agent to admit its limits without fear of punishment. In our custom AI-powered software projects, we incorporate safety layers ranging from mechanical verification to drift prediction. The safety prompt is only the first layer (L0), answering the question: 'Am I safe to admit I cannot verify this?' Without that layer, the upper layers (filesystem verification, token probability, format engineering, drift trends) face an internal adversary: a model incentivized to generate plausible outputs to satisfy execution gates.
In practice, many organizations working with cloud AWS or Azure deploy models with configurations that prioritize speed and completeness. The safety prompt can be easily integrated into the inference pipeline without changing infrastructure, and its cost is virtually zero (pennies per experiment). But its effect depends on context: if the environment is already safe, the prompt adds little; if the environment pressures toward fabrication, it can make a difference. That is why we always recommend stress-testing with questions the model cannot answer, as part of a cybersecurity strategy in AI: ensuring the system does not generate false information under pressure.
Another important dimension is business analytics. When integrating language models with BI and Power BI tools, agent honesty is critical. An AI-generated report that fabricates figures or trends can lead to wrong decisions. A safety prompt, combined with a real-data verification layer, allows the assistant to say 'I do not have access to that data' instead of inventing an answer. At Q2BSTUDIO we have developed process automation solutions that include such safeguards, ensuring that artificial intelligence acts as a reliable collaborator, not a fiction generator.
The experiment also taught us a methodological lesson: never interpret aggregate statistics without disaggregating. If we had stopped at the negative logprob average, we would have erroneously concluded that the safety prompt 'makes the model less confident.' By breaking down per question, we discovered the perfect positive correlation. This principle applies to any model testing: it is necessary to isolate the cases where change actually happens. In developing AI agents for clients, we apply this per-probe approach to ensure that every behavioral improvement is backed by greater internal confidence, not less.
Finally, one might ask: is a safety prompt enough to solve the honesty problem in AI? The answer is no. It is a tool within a broader ecosystem that includes prompt engineering, fine-tuning, external verification, and above all, a design culture that values authenticity over the appearance of competence. At Q2BSTUDIO we believe the future of artificial intelligence lies in systems that know how to say 'I don't know' as naturally as they answer trivial questions. Our custom software development services include these trust layers, from the cloud to the front-end, because true intelligence is not the one that always has an answer, but the one that recognizes its limits.
