We already know how to create an AI agent and the advantages it has for your company, but what happens when you create it and it doesn't work as expected? The mistakes it makes can be fixed with proper training.
Training an AI agent involves configuring its instructions, its documentation, and its integrations with other systems to achieve its intended purpose. Remember, an AI agent is software that acts intelligently without a human telling it every step to take, and for this, it's crucial to provide it with the correct guidance.
Steps to follow to train a conversational AI agent from scratch
1. Review how you created your AI agent
- Does the agent have a clear goal?
When something doesn't work, it's best to look back. To correctly create an AI agent, it's essential to be clear about the agent's mission, why you created it. It must have a specific function: you cannot create an agent that serves users wanting to know more about your company and also manages internal invoices. These are two very different functions that should be in two distinct agents.
- Are the integrations too limited or open?
Agents should be connected to your systems or tools to execute necessary tasks. For example, if you create an agent to manage existing customers, it should be connected to those customers' records. However, you should be aware of limitations because many errors can emerge from them.
In the previous example, the agent should have a limitation to only view and send information regarding the customer it is currently speaking with and, under no circumstances, should it provide data about other customers. This is the type of scenario you need to review in your integrations: what information the agent can access, when and with whom it can share it, and how and where it logs the information provided by the user it's interacting with. All of this depends on a quality integration.
- What platform did you use to create your AI agent?
There are multiple platforms and tools available to create an AI agent, for instance, if you have Microsoft, you might create it yourself using Copilot. There is also the alternative of creating fully customized AI agents for your company with tailored software. Each option, whether accessible online tools or custom software, has its advantages and disadvantages; choose what suits your company in the long term.
Comparative table:
2. Provide it with the information it needs
The agent needs to memorize what it has to apply later. For example, if you create an agent to resolve technical issues with your products, you must provide it with all documentation for each of your products and their technical specifications so it can understand the customer’s problem and find a possible solution.
These documents, in PDF or your preferred format, will be its knowledge base, so make sure there are no errors or contradictions, or the agent will fall into them.
3. Tell it how to behave
In general, training an artificial intelligence agent is like teaching a new colleague how to work: with the documentation you explain what the company is about and the products or services it offers, and with the instructions you outline its functions. But behavioral instructions require going a step further.
As with any new colleague, you will give clear instructions on what they should do: “You are now the customer service expert in my company,” “you need to provide users with information about our services and resolve their queries,” etc., but you must also consider things you might take for granted with a person, such as speaking concisely, addressing customer concerns first before offering or asking anything, not insulting or engaging in confrontations... Don’t take anything for granted; an AI agent doesn't know how to behave socially unless you tell it, which could lead to undesired outcomes.
Additionally, providing examples of what it can say will be very helpful. And something that is often overlooked: you need to be clear about who its target audience is to provide guidance tailored to them. For instance, if the audience is the general public, you should instruct it not to use technical jargon when speaking.
4. Test it extensively before publishing
If there is something crucial in training an AI agent, it is the testing phase. This will be the most time-consuming part and where you can detect unforeseen errors. Communicate with it as if you were a real user and continue the conversation to check if it performs the expected actions. It's likely you will have to resolve some of the following errors.
The 5 most common errors of AI agents that can be solved with proper training
1. Information hallucinations
The most common problem with all AI is making up answers, or what are known as hallucinations. It usually occurs when it is asked for information it does not have an answer to. This can be solved by instructing it on what to do when it lacks an answer: “do not make up this information, if you do not know how to respond to something because you lack the knowledge, say that you cannot assist with that topic and transfer the conversation to a team member through your connector.” This is an actual example of an instruction that solved the hallucination problem for one of our agents.
If, after giving these instructions, it continues to make up a particular response, review the documentation and other instructions to ensure there are no errors or contradictions.
2. Actions not executed
The goal for many agents is to execute some action, like scheduling an appointment. Your AI agent might know perfectly well how to carry the conversation with the customer, ask for the preferred time and day, and check for availability, but when it comes down to it, you do not see the appointment in your calendar.
If this is the case, task execution failures are typically due to poor integration with the necessary tools. Check that the agent has access to all systems or platforms involved in that action and that there are no unnecessary filters or limits.
3. Discussing its configuration
“I can't find that product in my documentation,” “I'm sorry, but my instructions tell me otherwise,” are responses your agent might give if you don't instruct it not to talk about its configuration, its knowledge documents, or the prompt it was created with.
If you don't want to disclose how you created and trained your AI agent, I suggest making it clear in its instructions.
4. Errors invisible to you
After extensive testing, you might think the agent is ready, but as soon as it’s deployed, a colleague from another department spots several errors. It is crucial that other people test your agent to generate scenarios you might not have thought of, especially if the AI agent is for another department.
For instance, have you instructed it to always respond in the language it is spoken to in? Because it might be a great agent in Spanish, but not in English.
5. If nothing works
If you've tried all the solutions we suggest and others, and it still makes mistakes, it might be due to one of these two reasons. You might be relying too much on AI: it's nearly impossible to anticipate all situations that can arise in a department, so it's always best to give your agent the option to pass the conversation to an employee when it does not know how to resolve it.
Or perhaps the agent you tried to create is too complex: there are many branches in your instructions depending on what the user says, many connected tools... Simplicity is always better for many reasons. It might be ideal to split that agent into several agents with simpler missions.
Extra: Data protection
The error few notice: being consistent with data protection legislation. Just like when someone registers in a form with their contact details must confirm that they have read and agree with the privacy policy, if your AI agent is going to collect and process personal data, it must obtain the appropriate consent according to regulations.
For example, you can instruct the agent to inform the user that their data will be used, send the privacy policy link, and ask if they agree before proceeding.
In summary, most errors can be detected by conducting sufficient real tests of the agent. To solve them, instruct the agent on what to do in that situation in its instructions or review the documentation and integrations, where there may be errors, contradictions, or improperly set limits.
How can I anticipate what errors my AI agent will make?
Testing. We wish we could give you a simpler answer, but the reality is that only trial-and-error and experience will help you understand how an AI behaves, which is not the same as a person.
At SEION, we've come to these conclusions after deploying dozens of AI agents for our clients. We have trained agents for very diverse functions within companies from various sectors. From agents dedicated to qualifying leads for the Marketing department of an industrial manufacturing company, to agents that converse with clients wishing to schedule or change appointments in a hair salon franchise.
If you want to clarify any doubts about how SEION's AI agents can help your particular company, you can always schedule a no-cost, no-obligation call here.
