The Intent Analyzer provides immediate feedback on the quality of example sentences and Intents within the Cognigy NLU model, enabling you to optimize AI Agents for better understanding and performance.
Key Benefits
- Rapid Quality Assessment. You can evaluate the NLU model’s effectiveness at a glance without extensive manual testing.
- Targeted Improvements. The Intent analyzer includes detailed scoring that highlights specific areas for refinement, thereby improving Intent recognition.
Restrictions
- After editing example sentences, the NLU model must be retrained to update scores.
- Strong overall Intent model scores don’t guarantee perfect individual Intents; red or yellow Intents may still need work.
- An Intent requires at least five example sentences. If trained with fewer, a negative feedback indicator will appear after the build, signaling that while the Intent still functions, its performance will be significantly reduced.
How to Set up
- In the Flow editor, go to NLU > Intents.
- Add a variety of example sentences into each Intent.
- In the upper-right corner, click Build Model to process the sentences and generate the initial model.
- View scores for the overall model, individual Intents, and example sentences.
Explore Intent Analyzer
The Intent Analyzer assesses three main components:- Overall Intent Model — shows a total score for the NLU model’s quality.
- Individual Intent — shows scores for each Intent, which reflect how well the NLU model recognizes them and if they overlap with other Intents. You will see the NLU score along with names of any overlapping Intents.
- Individual Example Sentence — each example sentence receives a score based on how effective it is within the NLU model. You can see the score and any overlapping Intents by hovering over the traffic light in the interface.
| Color | Score Range | Description | Recommendations |
|---|---|---|---|
| Green | Score > 0.8 | A green score indicates high confidence in Intent recognition. The NLU model is performing well and is ready for further testing by users. | Note that a perfect NLU model — where all sentences score green — isn’t necessary, the traffic light system helps quickly identify areas that may need attention and refinement. |
| Yellow | 0.8 > Score ≥ 0.6 | A yellow score indicates moderate confidence. The NLU model is performing well but shows that the Intent recognition could be more consistent. | Review example sentences with yellow scores. Add more varied examples or adjust poorly scoring examples to cover additional variations. Monitor Intents over time for any further inconsistencies. |
| Red | Score < 0.6 | A red score indicates significant issues with Intent recognition. The NLU model may fail to recognize the intent accurately, and urgent improvements are needed. | Review the training data for this intent. Focus on revising or expanding example sentences to better represent the variety of ways users might express the query. Consider re-training the NLU model with additional or revised data to improve recognition. |
How to Test
You can test NLU scores using the following interfaces:- GUI
- API
In the Interaction Panel, activate the debug mode feature and explore the detailed results in the
input.nlu.intentMapperResults.scores object.