Simple Framework for Interpretable Fine-Grained Text Classification

Abstract

Fine-grained text classification with similar and many labels is a challenge in practical applications. Interpreting predictions in this context is particularly difficult. To address this, we propose a simple framework that disentangles feature importance into more fine-grained links. We demonstrate our framework on the task of intent recognition, which is widely used in real-life applications where trustworthiness is important, for state-of-the-art Transformer language models using their attention mechanism. Our human and semi-automated evaluations show that our approach better explains fine-grained input-label relations than popular feature importance estimation methods LIME and Integrated Gradient and that our approach allows faithful interpretations through simple rules, especially when model confidence is high.

Appendices

A Model Fine-Tuning

For each of the three datasets (BANKING77, CLINC150, and HWU64), we train two models: a conventional single-sentence (1sent) model and a sentence-pair (2sent) model. The default initial model checkpoint (which we fine-tune) in the main body of this paper is BERT-​base [6], distributed on Huggingface’s Transformers library [42] with apache-​2.0 license. The datasets BANKING, CLINC, and HWU were accessed from Amazon Alexa AI’s DialoGLUE benchmark’s repository [22] with CC-BY-4.0, CC-BY-SA 3.0, and CC-BY-SA 3.0 licenses respectively.³

For both formulations, we fine-tuned BERT-​base on all examples in the training splits and evaluated on the test sets. The training script was run on NVIDIA Quadro T1000 4GB GPU, which allowed for a batch size of 8 (for both 1sent and 2sent) for the BERT-​base model with nearly 110 million parameters. The 1sent models took approximately 20 min to train on average across the three datasets, while the 2sent models took approximately one hour on average. For each dataset, we optimized using an Adam optimizer [11] with learning rate $3 \times 10^{-5}$ for 3 epochs, with the rest of the training hyperparameters set to default (by the Transformers library version 4.8.2).

Given an example x from the test or the training set, the 1sent model $m_1$ calculates |C| (the number of intents in the set of all candidate intents C) prediction scores $[y_i] \in \mathbb {R}^{|C|}$ and outputs the index i with the highest prediction score. On the other hand, the 2sent model $m_2$ outputs a match score ($m_2$ makes a binary prediction; we treat the prediction score for the positive class as this match score) given x and the natural language name $c_i$ of the intent concatenated together with a special separator token “[SEP]”. During the evaluation, we generate |C| inputs per x, pairing x with each intent $c_j$, and identify the intent with the highest match score as the 2sent prediction.

However, during training, it is not required to pair an example x with all intents. Therefore, given an example x annotated with an intent $c_{true}$, we generate $2 \times N + 1$ inputs, where $N < |C|$. The N of those are x paired with N random intents (negative labels) that are not $c_{true}$. Another N are random N example texts from the training split that do not have the intent $c_{true}$ (negative examples), paired with $c_{true}$. We set N equal to 5 to limit training time according to our resource constraints.

Table 5 reports the accuracy achieved by our main sentence-pair model (BERT-2sent) along with benchmark performances (BERT-fixed and BERT-tuned) from [3] and the accuracy of a single-sentence BERT model (BERT-1sent) that we trained for reference. BERT-2sent is on par with the benchmark model BERT-fixed⁴ and approaches BERT-1sent in accuracy (Table 4).

Table 4. Overview of the datasets

Table 5. Accuracy on the three datasets

B Crowdsourcing Details

Our study design followed those of previous work [29, 40]. We presented crowdworkers (on Prolific) with two alternative types of explanations and asked them to select their preference. We screened participants based on their English language proficiency and location (also as a proxy of language proficiency following [40]) because our evaluation datasets were in English, and based on their education level (at least undergraduate level) because our study involved non-trivial language understanding and familiarity with visualizations. We employed participants from the United Kingdom, the United States, Canada, Australia, and New Zealand with a minimum approval rate of 98% and a minimum of 100 previous submissions on Prolific. All submissions were anonymous and all participants were presented with an information sheet providing the details of our study and were asked for their consent at the beginning of our study.

Quality Control. Before our main head-to-head comparisons, we selected participants based on two preliminary rounds of questions. In the first preliminary round, we presented the crowdworkers with 12 multiple-choice questions based on example texts from fine-grained classification datasets (intent recognition and sentiment analysis) in order to test their ability to understand the fine-grained differences between English texts. We only selected participants who answered at least 11 questions correctly. In the second preliminary round, we presented the participants with 12 explanations, half of them feature importance and the other half feature relation (ours), and asked them to rate how well those explanations explained the correspondence between a given text and its label. We acknowledged that subjectivity can play a significant role here and participants can disagree with us on non-trivial cases or be more or less generous than us (but with similar relative ratings of the different explanations). Therefore, we passed participants through this round based on manually checking their submissions mostly for only low-quality responses (same choice for all questions, too short time spent per question, obvious random clicking on trivial cases, etc.).

Fig. 9.

User-interface of our head-to-head comparison study (example)

Payment. Our two preliminary rounds paid £1 (20£/h) and £1.5 (15£/h) for 3 and 6 min respectively. Our main survey was expected to take 8–13 minutes and participants were paid £3 (at least 12£/h).

User Interface. Figure 9 shows the user interface of our study, which is implemented on Qualtrics.

C Further Evaluation

In this section, we report the results from our initial (less computationally expensive) evaluation setting, which we refer to as the lite evaluation setting. The results from this setting are on both full test splits and the error sets (the subset of misclassified examples) of the three datasets BANKING77, HWU64, and CLINC150. The key difference from the main evaluation setting is that the lite setting asks the contrastive question “Why $c_1$ rather than $c_2$?” only once per example whereas the main setting asks it for all possible values of $c_2$. Moreover, the lite setting only evaluates the subset of the rules.

Within the lite setting, for each example in the error set, we aim to explain why the model made an error, i.e., “Why $c_1$ rather than $c_2$?”, where $c_1$ is the predicted intent and $c_2$ is the correct (ground-truth) intent. On the full sets, our aim is the same for misclassified examples, but for correctly classified examples, it is to explain “why not $c_2?$”, i.e., “Why $c_1$ rather than $c_2$?” where $c_1$ is predicted and $c_2$ is an intent similar to $c_1$. For this work, we pick such $c_2$ from the set of all intents other than $c_1$, by the highest vector similarity between the intent names (using the spaCy library).

1.1 C.1 Faithfulness

Tables 6 and 7 show the lite reproduction accuracy of our rules on the error and full sets respectively. We used the parameter $K = 1$ in the concentration rule since reproduction rate has a strong negative and monotonic correlation with K (see Appendix C.4).

Table 6. Reproduction accuracy (error set)

Table 7. Reproduction accuracy (full set)

All rules consistently achieved higher reproduction accuracy than random guesses. Magnitude consistently outperformed concentration with an average of 62.8% on the error sets and 82.5% on the full sets. Combining magnitude and concentration (mag. + con.) achieved the highest values with an average of 89.1% on the error sets and 97.8% on the full sets. The highest reproduction accuracy scores were 98.7% and 98.6%, which were on the full sets of CLINC150 and BANKING77 respectively. These high values suggest that our explanations faithfully explain model predictions.

Table 8. Spearman correlations

1.2 C.2 Correlation between Confidence and Reproduction Accuracy

Figure 10 shows that the reproduction rate of our best rule mag. + con. increased up to perfect accuracy on the error sets and near perfect on the full sets as we filtered out evaluation examples with increasing confidence threshold. This was the case despite decreasing number of examples, i.e., a higher drop in accuracy with each failure (see Fig. 11).

Just like in our main results, there was a strong tendency for positive and monotonic correlation (see Table 8, in which most correlation values are close to +1) between the two variables: reproduction accuracy and confidence threshold. The error set of BANKING77 had a weak correlation (value near 0) against this tendency, but Fig. 10a shows that the reproduction accuracy increased even for this case, though non-monotonically.

1.3 C.3 Importance of Special Tokens

We also hypothesized that certain reproductions may have failed because our cross-sentence relations did not always fully capture model reasoning. Importantly, we ignored the special tokens “[CLS]” and “[SEP]”, which are known for aggregating the input and receiving high levels of attention [4], to focus only on human-readable tokens (see Fig. 4). There are three special tokens in each concatenated input that we feed into our model: “[CLS]” as the first token, and two “[SEP]” tokens (one for delimiting the input pairs and one at the end of the input). We experimented with treating the “[CLS]” and the first “[SEP]” as features of text x, while treating the last “[SEP]” as a feature of intent $c_i$. This led to higher reproduction rates on all datasets (see Table 9).

Fig. 10.

Graphs showing the correlation between confidence threshold and reproduction accuracy

Fig. 11.

Graphs showing the correlation between confidence threshold and number of examples

1.4 C.4 Correlation Between Concentration Parameter K and Reproduction Accuracy

Figure 12 shows the relation between the parameter K (see Sect. 4.3) in our concentration rule and the reproduction accuracy (under the lite evaluation setting; see Appendix C.1) of the rule.

As Table 10 shows, there is a strong negative and monotonic correlation between the parameter K and reproduction accuracy.

Fig. 12.

Correlation between concentration parameter K and reproduction accuracy

Table 9. Reproduction accuracy (full set) when special tokens are treated as features of text x and intent $c_i$ (with confidence threshold) under lite evaluation setting (see Appendix C.1)

Table 10. Spearman correlations between K and reproduction accuracy. There is a strong negative correlation between K and reproduction accuracy across all datasets.

D Experiments with DeBERTa

Model training followed. Model training followed the same process for both DeBERTa and BERT except for a few details (see Appendix A). We used microsoft/deberta-​base [7] with approximately 140 million parameters, distributed on Huggingface’s Transformers library [42] with an MIT license. We used learning rates of $1 \times 10^-2$ or $2 \times 10^-2$ selected by greedy search (due to resource constraints), and batch size of 1 or 2 with our DeBERTa models. Training took approximately one hour for the 1sent models across the three datasets, and 4–6 h for the 2sent models. On the other hand, our main evaluation loop takes approximately less than 5 min on the full set for both 1sent and 2sent models. The DeBERTa model trained with single-sentence formulation (DeBERTa-1sent) achieved the highest accuracy among the models we trained. However, DeBERTa trained with the sentence-pair formulation (DeBERTa-2sent) achieved scores lower than BERT-2sent, possibly due to non-optimal choices of hyperparameters (especially learning rate). Nevertheless, we proceeded to use this DeBERTa-2sent model since its performance is adequate for our primary purpose of evaluating interpretability (Table 11).

Table 11. Accuracy of our fine-tuned models including DeBERTa trained with single-sentence (DeBERTa-1sent) and sentence-pair (DeBERTa-2sent) approaches

Faithfulness Results. The faithfulness results with DeBERTa-2sent were similar to those with BERT-2sent (under the lite evaluation setting; see Appendix C), since the mag. + con. rule achieved high reproduction accuracy on both the error and full sets (see Tables 12 and 13). However, an interesting difference was that magnitude was better at reproducing model prediction than concentration with BERT, but it was the other way around with DeBERTa.

Table 12. Reproduction accuracy (error set) with DeBERTa

Table 13. Reproduction accuracy (full set) with DeBERTa