Department of Computer, Control and Management Engineering

The recent explosion of deep learning techniques boosted the application of Artificial Intelligence in a variety of domains, thanks to their high performance. However, performance comes at the cost of interpretability: deep models contain hundreds of nested non-linear operations that make it impossible to keep track of the chain of steps that bring to a given answer. In our recently published paper [1], we propose a method to improve the interpretability of a class of deep models, namely Memory Augmented Neural Networks (MANNs), when dealing with sequential data. Exploiting the capability of MANNs to store and access data in external memory, tracking the process, and connecting this information to the input sequence, our method extracts the most relevant sub-sequences that explain the answer. We evaluate our approach both on a modified T-maze [2, 3] and on the Story Cloze Test [4], obtaining promising results.