Evaluating Consistency of XAI Methods in Hate Speech Detection
Technical evaluation of LIME and SHAP explanation consistency using the CardiffNLP RoBERTa model for automated content moderation transparency.
About
This research focuses on transparency in automated content moderation AI methods, examining how explainable AI methods perform across a hate speech detection model. The study leverages multiple datasets including HateXplain, MLMA, and Measuring Hate Speech to analyze consistency in the use of LIME and SHAP (XAI methods) used across platforms such as Twitter, YouTube, Reddit, and Gab.
Features
- LIME and SHAP: XAI methods with transformer-based CardiffNLP RoBERTa model
- Quantitative evaluation: Measured with multiple consistency metrics for cross-method comparison
- Stratified sampling: Samples across diverse prediction scenarios to ensure representative analysis
- Comprehensive visualizations: Developed for detailed explanation analysis and interpretation
Tech Stack
Python
Transformers (Hugging Face)
LIME & SHAP
Pandas
NumPy
Matplotlib
Seaborn
Technical Skills
- Explainable AI: Integration and evaluation of state-of-the-art explanation techniques for black-box models
- Data Analysis: Comprehensive exploratory data analysis and statistical evaluation of results
- Data Visualization: Creation of informative and intuitive visualizations to communicate complex findings
- Research Methodology: Design and implementation of rigorous experimental protocols to test specific hypotheses
- Python Programming: Extensive use of Python and relevant libraries for data science and machine learning tasks
Links
Visualizations
