Attribution & Model-agnostic Tools
9 entriesExplain predictions for any model.
01DeepLIFTDeep Learning Important FeaTures — assigns importance scores to input features by comparing the activation of each neuron to its reference activation. Last commit on the default branch was in November 2021.
02LIMELocal Interpretable Model-Agnostic Explanations — explains predictions of any machine learning classifier by learning a locally faithful interpretable model around each prediction. Last commit on the default branch was in July 2021.
03SHAPSHapley Additive exPlanations — a game theoretic approach to explain the output of any machine learning model, connecting optimal credit allocation with local explanations.
04ELI5Python library for visualizing and debugging various machine learning models using a unified API — supports scikit-learn, XGBoost, LightGBM, and more.
05OmniXAIOmni eXplainable AI Python library offering omni-way explainability and interpretable ML capabilities — addressing many pain points in explaining ML model decisions across tabular, text, image, and time-series data.
06DALEXX-rays any model and helps explore and explain its behaviour — providing consistent tools for understanding how complex black-box models produce their predictions.
07Alibi ExplainOpen source Python library for machine learning model inspection and interpretation — providing high-quality black-box, white-box, local, and global explanation methods for classification and regression models.
08CaptumModel interpretability library for PyTorch.
09DiCEDiverse counterfactual explanations.
Toolkits, Dashboards & Visual Tools
8 entriesInteractive and broader suites.
10What-If ToolVisually probe the behavior of trained machine learning models with minimal coding — explore model performance, fairness, and counterfactual examples interactively.
11Rulex PlatformMachine learning capabilities that ensure essential transparency, peak performance, and unwavering accuracy throughout the entire forecasting process — with native XAI built in.
12Activation AtlasesUses feature inversion to visualize millions of activations from an image classification network — creating an explorable atlas of features the network has learned to represent concepts.
13Ethical Institute XAI FrameworkIntroduces explainability and bias evaluation in AI systems using a tool-plus-process approach — going beyond algorithms to operationalize responsible AI practices.
14InterpretMLA toolkit to help understand models and enable responsible machine learning — with Explainable Boosting Machine (EBM) and support for SHAP, LIME, and other explainers.
15ShapashPython library making machine learning interpretable and accessible to everyone — with clear, explicit visualization labels that facilitate understanding for both technical and non-technical audiences.
16TCAVTesting with Concept Activation Vectors — interpretability method to understand what signals neural network models use for prediction using human-friendly concepts. Last commit on the default branch was in September 2021.
17ExplainerDashboardLibrary for quickly building interactive dashboards to analyze and explain ML model predictions and workings — supporting scikit-learn, XGBoost, CatBoost, and LightGBM models.
Books & Guidance
3 entriesReading and policy references.
18Interpretable Machine Learning (Christoph Molnar)Free online book on interpretable machine learning.
19NIST AI Risk Management FrameworkNIST framework that includes explainability and interpretability among trustworthy AI characteristics.
20DARPA Explainable AI (XAI) programDARPA research program that helped establish the XAI field.
Related Pages
6 entriesNeighbouring pages on this site.
21Python Libraries for Explainable AIPython XAI libraries.
22ML Model Monitoring ToolsMonitoring models.
23Machine Learning FrameworksML frameworks.
24Deep Learning FrameworksDeep learning frameworks.
25Machine LearningMachine learning hub.
26AI Explainability Techniques, Tools and LibrariesTechniques, mechanistic interpretability, evaluation of explanations, guidance and libraries.
Frequently Asked Questions
What is explainable AI?
Explainable AI (XAI) covers methods that make a machine learning model's predictions understandable to people, for example by showing which features drove a result.
Which framework should I use?
SHAP and LIME are the most widely used model-agnostic tools, Captum suits PyTorch networks, and InterpretML offers glass-box models and explainers in one toolkit.
Are all of these projects active?
No. LIME, DeepLIFT and TCAV have had no commits on their default branches since 2021, so check each project's activity before relying on it.
How current is this page?
Links were checked in October 2026. Sites that block automated checks are kept and may need a manual click. Last-commit dates come from GitHub in October 2026 and show default-branch activity only; check each project's release notes before adopting it.