Looking under the Hood of AI Conversational Search - Explainability of AI output explored
- Presentation
- Artificial Intelligence (AI) in Technical Communication
- 10. November
- 11:30 AM (CET) - 12:15 PM (CET)
- Plenum 1
-
Dr Gesine Stanienda
- SAP SE
-
Michael Pflanz
- SAP SE
Contents
"That's not what I was asking about!" is a common phrase we hear when we observe users' reactions to answers from LLM-based agents. To understand how and why LLM-based agents deal with user utterance the way they do, it is important to make answers explainable.
Explainability can be achieved in several ways, ranging from simply showing the ground-truth sources to the user to asking the agent to explain its reasoning.
In addition to generating explainability data, it is also important to present the data in a meaningful and user-centric way. To this end, explainability towards end-users needs to be aligned both with UX and development teams as well as the QA team monitoring and analyzing the explainability measures. On the other hand, development teams require mass tests with quick turn-around times to see how the agent performs against thousands of queries to catch inconsistencies across all use cases. Here, the QA team needs to devise test cases as well as implement the relevant infrastructure to expose the inner workings of the LLM-based agent quickly and thoroughly in an iterative approach.
Takeaways
We will be showing different approaches, both end-user-facing as well as development- and QA-facing, that allow us to look under the hood and get information that helps us shine a light on what happens in the black box that are LLMs
Prior knowledge
Basic understanding of LLM-based agents is helpful.
Knowledge of Q-Gates for LLMs can also help.