Explainable AI in Your Thesis or Paper: A Practical Guide

Explainable AI (XAI) has moved from a "nice extra" to something examiners, reviewers and clinical or industrial partners routinely ask for. If your thesis or paper uses a deep learning model for diagnosis, fault detection, crop classification or network intrusion detection, you will almost certainly be asked: why did the model make that decision? This guide walks through how to choose an explanation method, how to apply it carefully, and how to report it so that it strengthens your work instead of becoming a decorative figure.

1. Start with the question, not the tool

Many students open a notebook, run SHAP or Grad-CAM, and paste the output into the results chapter. The stronger approach is to decide first what you need the explanation to answer. Typical questions include:

• Global behaviour: which input features matter most to the model overall?
• Local decisions: why was this particular patient, image or network flow classified the way it was?
• Debugging: is the model relying on a shortcut, such as a hospital watermark, image border or dataset artefact?
• Trust and adoption: can a domain expert check that the reasoning matches known physics, physiology or engineering knowledge?

Write the question down in your methodology. It tells the reader why the explanation exists and gives you a criterion for judging whether it worked.

2. Match the method to your data and model

There is no single best XAI method. A practical shortlist looks like this:

• Tabular data (clinical records, sensor features, energy consumption): SHAP values are widely used for both global and local explanations, and permutation feature importance is a simple, model-agnostic baseline. LIME offers local surrogate explanations but can be sensitive to its sampling settings.
• Images (medical scans, plant phenotyping, defect inspection): Grad-CAM and its variants produce class-specific heatmaps for convolutional networks. Integrated Gradients and other attribution methods work at the pixel level. For Vision Transformers, attention maps are often shown, but treat raw attention with caution, because attention weights are not automatically faithful explanations.
• Time series and signals (ECG, gait sensors, acoustic emission): attribution methods applied over time steps, or occlusion of time windows, can show which segments drive a prediction.
• Inherently interpretable models: sometimes a well-tuned decision tree, generalized additive model or logistic regression is accurate enough. Reporting that comparison honestly is itself a valuable contribution.

Justify your choice in one or two sentences, and cite the original paper for every method you use.

3. Apply it carefully

XAI outputs are easy to produce and easy to misread. A few habits make a big difference:

• Explain the final model, trained and evaluated exactly as reported, not an earlier experimental version.
• Use held-out data for the examples you show, so the explanation reflects behaviour on unseen cases.
• Show correct and incorrect predictions. Explanations of misclassified cases are often the most informative part of the analysis.
• Avoid cherry-picking. State how the displayed examples were selected, for example randomly or one per class.
• Record settings such as background dataset size for SHAP, the target layer for Grad-CAM, or the baseline for Integrated Gradients. These choices change the result.

4. Check that the explanation is trustworthy

A heatmap that "looks right" is not evidence on its own. Reviewers increasingly expect at least a basic sanity check. Useful options include:

• Randomization checks: if you randomize the model weights and the explanation barely changes, the method is not telling you much about what the model learned.
• Deletion or insertion tests: remove the most important features or regions and observe whether model confidence drops as expected.
• Consistency across methods: if SHAP and permutation importance disagree strongly, discuss why.
• Expert review: where possible, ask a clinician, engineer or agronomist whether highlighted regions or features are plausible, and describe that process honestly.

5. Report it clearly

A good XAI section in a paper or thesis usually contains:

1. A short motivation linked to the question from step 1.
2. The method, library and key settings, with enough detail for replication.
3. A global view, such as a feature-importance plot or class-averaged heatmaps.
4. A small, fairly selected set of local examples with captions that say what the reader should notice.
5. Sanity checks and their outcome.
6. A limitations paragraph: explanations describe model behaviour, not causal truth, and they can be unstable.

Keep figure captions self-contained. A reviewer skimming your paper should understand each heatmap or plot without hunting through the text.

6. Common mistakes to avoid

• Claiming that an explanation "proves" the model is correct or clinically safe.
• Presenting attention maps as if they were guaranteed explanations.
• Showing only the most convincing examples.
• Using XAI to describe correlations as if they were causes.
• Leaving out the software version and settings, which makes the results hard to reproduce.

Bringing it together

Done well, explainability turns a black-box accuracy number into an argument that examiners and reviewers can follow and test. It also protects you: if your model is relying on a shortcut, it is far better to discover that during your own analysis than in a reviewer's report.