Modified Transformer Architecture to Explain Black Box Models in Narrative Form

Modified Transformer Architecture to Explain Black Box Models in Narrative Form

Diksha Malhotra, Poonam Saini, Awadhesh Kumar Singh
Copyright: © 2022 |Pages: 19
DOI: 10.4018/IJSWIS.297040
Article PDF Download
Open access articles are freely available for download

Abstract

The current XAI techniques present explanations mainly as visuals and structured data. However, these explanations are difficult to interpret for a non-expert user. Here, the use of natural language generation (NLG)-based techniques can help to represent explanations in a human-understandable format. The paper addresses the issue of automatic generation of narratives using a modified transformer approach. Further, due to the unavailability of a relevant annotated dataset for development and testing, the authors also propose a verbalization template approach to generate the same. The input of the transformer is linearized to convert the data-to-text task into text-to-text task. The proposed work is evaluated on a verbalized explained PIMA Indians diabetes dataset and exhibits significant improvement as compared to existing baselines for both manual and automatic evaluation. Also, the narratives provide better comprehensibility to be trusted by human evaluators than the non-NLG counterparts. Lastly, an ablation study is performed in order to understand the contribution of each component.
Article Preview
Top

Introduction

With the advancement in technology, Artificial Intelligence (AI) has gained enormous popularity and applicability in various domains such as healthcare, finance, retail, etc. (Sarivougioukas & Vagelatos, 2020). In order to produce high-performance commercial products, many AI-based companies tend to develop predictive models whose behaviour may sometimes deviate from human expectations (Cheng et al., 2021). Traditional AI systems often lack transparency in decisions due to their complex nature and hence, are unable to explain such deviations (EU, 2019). In critical systems like autonomous (Fiorini, 2020; Pandey & Banerjee, 2019) and AI-assisted healthcare systems (Gupta et al., 2021; Sun et al., 2019), there is a need to induce explainability of decisions for social, practical, and legal reasons. Hence, recently the branch of eXplainable Artificial Intelligence (XAI) has gained importance in applications where the result of committing a mistake can be disastrous (Gunning & Aha, 2019). It refers to the branch of AI which provides reasoning behind the predictions of any AI model. Further, XAI techniques can be broadly divided into intrinsic and post-hoc wherein, intrinsic techniques aim to provide explanation along with the prediction. Whereas the post-hoc techniques are applied on models to produce explanation after the output is predicted. The paper aims to build an explanation-to-narration module for post-hoc explanations.

The current state-of-art XAI techniques present explanations in many forms such as visual, audio, linguistic, tabular, etc. The traditional trend in the literature is to represent results in the form of visuals, especially heat maps. However, these may not be well understood by a non-technical user. Out of the above-mentioned ways, linguistic methods can be attractive for interested non-expert users (J.M. Alonso et al., 2020). These allow users to understand the model’s predictions without any mathematics or engineering background and instigate willingness among them to use autonomous systems (J.M. Alonso et al., 2020). To date, few works have directly addressed the possibility of generating textual explanations from the structured output of an explainer. However, the NLP community (Singh & Sachan, 2021), especially working in data-to-text generation, can add a linguistic layer to many of the state-of-art post-hoc XAI systems proposed so far (Fayoumi & Hajjar, 2020; Inan & Dikenelli, 2021).

The explanations generated by the state-of-art post-hoc local XAI techniques such as LIME (Ribeiro et al., 2016), SHAP (Roth, 1988), etc. are generally in the form of IJSWIS.297040.m01IJSWIS.297040.m02 where IJSWIS.297040.m03 and IJSWIS.297040.m04 represent the features and their contribution respectively for each instance IJSWIS.297040.m05. Natural Language Generation (NLG) techniques can convert data into text or text into text depending on the application requirement (Reiter & Dale, 1997). A sub-field of NLG i.e., data-to-text generation can be employed on the structured explanation IJSWIS.297040.m06 to generate the corresponding narrative.

Figure 1.

An example of the structured explanation and corresponding narrations

IJSWIS.297040.f01

Complete Article List

Search this Journal:
Reset
Volume 20: 1 Issue (2024)
Volume 19: 1 Issue (2023)
Volume 18: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 17: 4 Issues (2021)
Volume 16: 4 Issues (2020)
Volume 15: 4 Issues (2019)
Volume 14: 4 Issues (2018)
Volume 13: 4 Issues (2017)
Volume 12: 4 Issues (2016)
Volume 11: 4 Issues (2015)
Volume 10: 4 Issues (2014)
Volume 9: 4 Issues (2013)
Volume 8: 4 Issues (2012)
Volume 7: 4 Issues (2011)
Volume 6: 4 Issues (2010)
Volume 5: 4 Issues (2009)
Volume 4: 4 Issues (2008)
Volume 3: 4 Issues (2007)
Volume 2: 4 Issues (2006)
Volume 1: 4 Issues (2005)
View Complete Journal Contents Listing