Imagine a customer asking an AI agent whether a product can integrate with their company’s system. The agent cannot find enough documentation, but confidently replies: “Yes, the integration is fully compatible.” The conversation moves forward, the customer buys, and during implementation the team discovers that compatibility depended on a feature that does not exist.

This hypothetical example reveals a problem that goes beyond incorrect information: the agent has turned a gap in its knowledge into a commercial promise. In an immediate evaluation, the service might look excellent, with a fast response, a friendly tone, and no apparent difficulty. The damage would emerge later, when someone else had to deal with the distance between what was promised and what could actually be delivered.

Teaching AI agents to say “I don’t know” means preventing invented certainty from filling that gap. To work, however, this guidance must influence training, evaluation, and the actions the system can execute.

What does it mean to teach AI to recognize its limits?