NLU Performance¶
The NLU Performance Dashboard provides valuable insights into the performance of your AI Agent's natural language understanding (NLU) capabilities. It allows you to monitor and enhance message comprehension, Intent recognition, and execution efficiency. Moreover, it offers easy access to resources like the Intent Trainer for further performance improvement.
You can find more information about our NLU and Intents in the NLU Overview article.
The NLU Performance data of your AI Agent is visualized in the following charts:
These charts display all data based on the global filter and local filter settings.
Indicators¶
The Indicator charts presents real-time data that refreshes every 30 seconds.
Understood Messages¶
Displays the percentage of messages understood during conversations. You can find a calculation example with data visualization in the Understood/Misunderstood bar chart.
Average Execution Time¶
Displays the average execution time across a specific timespan.
Formula:
Average Execution Time = Sum of Execution Times / Total number of inputs
You can find a calculation example with data visualization in the Average Execution Time line chart.
Avg Intent Score¶
Displays the average Intent score for the selected time frame. Higher Intent scores indicate better AI Agent performance in matching user queries. For more information, see Machine Learning Intents.
To find the average intent score, Insights sums up all the intent scores and divides the sum by the total number of records.
Source table:
| Analytics Record | Intent Score |
| ---------------- | ------------ |
| Record-1 | 0.30 |
| Record-2 | 0.90 |
| Record-3 | 0.10 |
| Record-4 | 1 |
| Record-5 | 0.20 |
| Record-6 | 0.02 |
| Record-7 | 0.50 |
- Sum of Intent Scores =
0.30 + 0.90 + 0.10 + 1 + 0.20 + 0.02 + 0.50 = 3.02 - Number of Records =
7 - Average Intent Score =
(Sum of Intent Scores) / (Number of Records) - Average Intent Score =
3.02 / 7
Result: 0.43
Max Intent Score¶
Indicates the number of Intents that were matched with 100% efficacy.
Source table:
| Analytics Record | Intent Score |
| ---------------- | ------------ |
| Record-1 | 0.30 |
| Record-2 | 0.90 |
| Record-3 | 0.10 |
| Record-4 | 1 |
| Record-5 | 0.20 |
| Record-6 | 0.02 |
| Record-7 | 0.50 |
Result: In the provided source table, there is one record with an intent score of 1, so the max intent score is 1.
Bar charts¶
Understood / Misunderstood Messages¶
Displays the percentage of messages understood and misunderstood by the system over a selected time frame.
A message is counted as understood if it has triggered an Intent, a slot is matched, or it is marked as understood via the Code Node or Overwrite Analytics Node.
A message is not counted as understood
if it is marked as Don't count in the Overwrite Analytics Node
or sent during an active handover
without configuring AI Copilot Whisper Flow.
By clicking 
,
you can select the Go to Intent Trainer,
which will take you to the application in Cognigy.AI for improving your Flow and Intents. For more information, see Intent Trainer.
Source table:
| Date | understood | misunderstood | Percentages |
| -----------| ------------ | --------------- | --------------- |
| 2023-09-13 | 20 | 33 | 38/62 |
| 2023-09-14 | - | - | - |
| 2023-09-15 | 29 | 18 | 62/38 |
| 2023-09-16 | 0 | 0 | - |
| 2023-09-17 | 0 | 0 | - |
| 2023-09-18 | 0 | 0 | - |
| 2023-09-19 | 0 | 0 | - |
| ------------------------------------------------------------- |
| TOTAL | 49 | 51 | |
Calculation for the selected timeframe:
- Misunderstood % =
(51 / 100) * 100 = 51% - Understood % =
(49 / 100) * 100 = 49%
Calculation example for the date 2023-09-15:
- Misunderstood % =
(18 / 47) * 100 = 38% - Understood % =
(29 / 47) * 100 = 62%
Result:
Top Intents¶
Displays the top-rated number of Intents. The summary view shows a maximum of 6 Intents, while the detailed view displays up to 25. Matching an Intent triggers its recording in the analytics data. The data is then grouped by the matched Intent and sorted in descending order. Source table:
| Intent | Total number |
| ----------------- | ------------- |
| Flight discounts | 12 |
| Book a flight | 5 |
| Change a flight | 5 |
| Additional fee | 4 |
| Reject a flight | 3 |
| Transfer flights | 3 |
Result:
Top Slots¶
Similar to Top Goals and Top Intents, this chart displays top-rated number of slots in a horizontal bar chart.
Source table:
| Slot | Number |
| ----------- | ----------- |
| DATE | 2 |
| NUMBER | 1 |
Result:
Intents by Score Range¶
Displays the scoring range of matched Intents. Each score range is displayed as a bar with minimum, maximum, and average values. Clicking on the three-dots menu item, you can select the Go to Intent Trainer that leads you to the application in Cognigy.AI to improve your Flow. For more information, see Intent Trainer.
Source table:
| Date | intentScoreAvg | intentScoreMax | intentScoreMin |
| ------------------ | -------------- | -------------- | ------------------ |
| cakes | 0.6517875252045072 | 0.6517875252045072 | 0.6517875252045072 |
| pizza | 0.6863160928710601 | 0.7237952611766103 | 0.6491475952551133 |
| drinks | 0.698537294767418 | 0.698537294767418 | 0.698537294767418 |
| ice cream | 0.7862064089630107 | 0.7862064089630107 | 0.7862064089630107 |
| transfer with refer| 1 | 1 | 1 |
| transfer with dial | 1 | 1 | 1 |
| play | 1 | 1 | 1 |
| DTMF | 1 | 1 | 1 |
Top Intent Scores¶
Indicates the top-rated numbers of Intents displayed in a bar chart. For more information, see Intent Analyzer.
Source table:
| Intents | Intent Score |
| -------------------- | ------------------ |
| play | 1 |
| transfer with dial | 1 |
| transfer with refer | 1 |
| DTMF | 1 |
| ice cream | 0.7862064089630107 |
| drinks | 0.698537294767418 |
| pizza | 0.6863160928710601 |
| cakes | 0.6517875252045072 |
Result:
- Intent Score Max:
1 - Intent Score Min:
0.65
Line charts¶
Average Execution Time¶
Displays the average time in milliseconds it takes to execute a Flow process over a selected time frame.
Formula:
Average Execution Time = Sum of Execution Times / Total number of inputs
Source table:
| Date | Execution Time |
| ------------ | --------------- |
| 2023-09-13 | 130 |
| 2023-09-14 | 0 |
| 2023-09-15 | 123 |
| 2023-09-16 | 0 |
| 2023-09-17 | 0 |
| 2023-09-18 | 0 |
| 2023-09-19 | 0 |
Result: (130ms + 123ms) / 2 = 127.5ms
Download Report¶
To download a report as a CSV file for the chart,
click 
in the upper-right corner of the chart.
To create a report file, refer to the Download Report article.