Questions / What does the agent need to know?
Why does my AI agent give confident wrong answers about my own business?
Because it is answering from what it learned about the world in general, not from what is true in your company. A model trained on the whole internet knows a lot about business in general and nothing about your contracts, your prices or your clients. Unless you give it your own information at the moment it answers, it fills the gap with something that sounds right.
The agent does not know that it does not know
A new employee who does not know your discount policy will usually say “let me check.” A language model usually will not. It is built to produce a fluent answer, and a fluent answer about a typical company reads exactly like a correct answer about yours. That is why the mistakes are dangerous: they do not look like mistakes.
This is not a defect you can wait for the next model to fix. A better model writes better sentences. It still has never seen your contracts.
The fix is to make it look before it answers
The principle is simple: before the agent answers a question about your business, it must first find the relevant facts in your own sources, and then answer from those facts. The industry name for this is retrieval-augmented generation, or RAG: retrieve first, then generate.
The name matters less than the habit it describes. A good agent behaves like a careful employee: it opens the contract before quoting a clause, checks the price list before quoting a price, reads the client’s history before describing the relationship.
What this changes for you as a leader
Three things follow from this principle, and none of them is technical:
- The quality of the agent depends on the state of your information. If your contracts are scattered across email attachments and personal folders, the agent will find some and miss others. Putting your knowledge in order is often the real first project.
- Every answer should show where it came from. An agent that says “according to clause 7 of the agreement signed in March” can be checked in a minute. An agent that simply states an answer cannot. Ask for the source every time; it is your cheapest protection.
- “I could not find it” must be an acceptable answer. Tell your team that an agent which admits a gap is working correctly, and an agent which always has an answer is the one to worry about.
The question to ask your team
Before any agent goes in front of staff or clients, ask one question: “When it answers about our business, where exactly does that answer come from?” If the reply is “from the model,” the agent is guessing. If the reply names your own documents and systems, you are on the right path.