The testing phase is key to success in the Virtual Assistant development process.
The quality of the conversations your users will have and the naturalness with which they will be able to resolve their queries is closely linked to the time, dedication, and effort put into this evaluation and training stage, because this is where you will be able to detect opportunities for improvement before the end user sees it.
The goal is to subject your virtual assistant to a scenario that is as close as possible to its actual use, so that the content loading team can adjust the knowledge already incorporated or generate new knowledge.
- As input, you will use the initial content with all the questions and answers worked on and uploaded to the platform, nothing outside that framework of information.
- The test should be carried out by a small group of end users or by company personnel in direct contact with them, such as the customer service department.
- Those conducting the tests should be familiar with the ways of expressing themselves, the vocabulary used, and, above all, the most frequent questions asked by customers.
- In addition, for these tests to be successful and contribute to a smooth rollout, they must also consider that the version used is a test version and is designed to find opportunities for improvement. Each finding prevents the same thing from happening later in the final version. In turn, the actual content uploaded to Virtual Assistant, the specific topics it will be able to respond to at launch, and the scope it covers.
- During testing, we suggest trying out different alternatives and ways of asking questions, spelling or typing errors, abbreviations, etc., always respecting the content uploaded to the platform.
- If the navigation assistance add-on (Assisted Navigation) is used in any response, keep in mind that when you change pages, the Virtual Assistant will disappear, and you will have to return to the original demo to continue the test. This is because the assistant is not yet placed on all the URLs of the site where it will be operational.
- It is also important to test the content on the different channels where you will implement the virtual assistant.
In terms of the working methodology, in addition to taking entire conversations as samples for analysis, we suggest paying special attention to two records:
- Unresolved questions: queries for which there is no answer. Virtual Assistant stores these questions in a separate section called Training, where they will be loaded as new ways of asking about existing content or will create new content.
- Poorly rated answers: each answer can be rated positively or negatively. Test administrators should use this tool by implementing the following criterion: negatively rate any answer that has been answered incorrectly. In this way, content managers can adjust the way questions are asked so that the correct answer is given.
These external tests may give rise to new combinations and structures of questions, alternatives for technical vocabulary, and words and abbreviations specific to the Virtual Assistant.
All these new ways of asking questions may mean that, in the first few attempts, the assistant does not correctly answer questions on the topics loaded. Therefore, the loading team will be on the lookout for opportunities for improvement to adjust the final details before launch, as it takes several months to achieve a solid and robust automated service tool.