Integrating Structural Causal Model Ontologies with LIME for Fair Machine Learning Explanations in Educational Admissions
DOI:
https://doi.org/10.62411/jcta.10501Keywords:
knowledge discovery in databases (KDD), structural causal model (SCM), ontology, local interpretable model-agnostic explanations (LIME), fairnessAbstract
This study employed knowledge discovery in databases (KDD) to extract and discover knowledge from the Benue State Polytechnic (Benpoly) admission database and used a structural causal model (SCM) ontological framework to represent the admission process in the Nigerian polytechnic education system. The SCM ontology identified important causal relations in features needed to model the admission process and was validated using the conditional independence test (CIT) criteria. The SCM ontology was further employed to identify and constrain input features causing bias in the local interpretable model-agnostic explanations (LIME) framework applied to machine learning (ML) black-box predictions. The ablation process produced more stable LIME explanations devoid of fairness bias compared to LIME without ablation, with higher prediction accuracy (91% vs. 89%) and F1 scores (95% vs. 94%). The study also compared the performance of different ML models, including Gaussian Naïve Bayes, Decision Trees, and Logistic Regression, before and after ablation. The limitation is that the SCM ontology is qualitative and context-specific, so the fair-LIME framework can only be extrapolated to similar contexts. Future work could compare other explanation frameworks like Shapley on the same dataset. Overall, this study demonstrates a novel approach to enforcing fairness in ML explanations by integrating qualitative SCM ontologies with quantitative ML/LIME methods.References
T. Calders and B. Custers, “What Is Data Mining and How Does It Work?,” in Discrimination and Privacy in the Information Society, vol. 3, B. Custers, T. Calders, B. Schermer, and T. Zarsky, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 27–42. doi: 10.1007/978-3-642-30487-3_2.
L. Cao and C. Zhang, “The evolution of KDD: Towards domain-driven data mining,” Int. J. Pattern Recognit. Artif. Intell., vol. 21, no. 04, pp. 677–692, Jun. 2007, doi: 10.1142/S0218001407005612.
M.-S. Chen, J. Han, and P. S. Yu, “Data mining: an overview from a database perspective,” IEEE Trans. Knowl. Data Eng., vol. 8, no. 6, pp. 866–883, 1996, doi: 10.1109/69.553155.
J. W. Seifert, “Data Mining: An Overview,” 2004.
D. T. Larose and C. D. Larose, Discovering knowledge in data, 2nd ed. Hoboken, NJ: Wiley-Blackwell, 2014.
N. Ye, The Handbook of Data Mining. CRC Press, 2003. doi: 10.1201/b12469.
P.-N. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Upper Saddle River, NJ: Pearson, 2005.
C. Rygielski, J.-C. Wang, and D. C. Yen, “Data mining techniques for customer relationship management,” Technol. Soc., vol. 24, no. 4, pp. 483–502, Nov. 2002, doi: 10.1016/S0160-791X(02)00038-6.
G. S. Linoff and M. J. A. Berry, Data mining techniques, 3rd ed. Nashville, TN: John Wiley & Sons, 2011.
O. P. Rud, Data mining cookbook. Nashville, TN: John Wiley & Sons, 2000.
E. W. T. Ngai, L. Xiu, and D. C. K. Chau, “Application of data mining techniques in customer relationship management: A literature review and classification,” Expert Syst. Appl., vol. 36, no. 2, pp. 2592–2602, Mar. 2009, doi: 10.1016/j.eswa.2008.02.021.
F. Guo and H. Qin, “Data Mining Techniques for Customer Relationship Management,” J. Phys. Conf. Ser., vol. 910, p. 012021, Oct. 2017, doi: 10.1088/1742-6596/910/1/012021.
P. S. Raju, V. R. Bai, and G. K. Chaitanya, “Data mining: Techniques for Enhancing Customer Relationship Management in Banking and Retail Industries,” Int. J. Innov. Res. Comput. Commun. Eng., vol. 2, no. 1, pp. 2650–2657, 2014.
R. Dass, “Data Mining in Banking and Finance: A Note for Bankers.” Indian Institute of Management Ahmadabad, 2006.
V. Jayasree and R. V. S. Balan, “A Review on Data Mining in Banking Sector,” Am. J. Appl. Sci., vol. 10, no. 10, pp. 1160–1165, Oct. 2013, doi: 10.3844/ajassp.2013.1160.1165.
K. I. Moin and Q. B. Ahmed, “Use of data mining in banking,” Int. J. Eng. Res. Appl., vol. 2, no. 2, pp. 738–742, 2012.
N. K. Hariharan, “Applications of data mining in finance,” Int. J. Innov. Eng. Res. Technol., vol. 5, no. 2, pp. 72–77, 2018.
I. Yoo et al., “Data Mining in Healthcare and Biomedicine: A Survey of the Literature,” J. Med. Syst., vol. 36, no. 4, pp. 2431–2448, Aug. 2012, doi: 10.1007/s10916-011-9710-5.
N. Jothi, N. A. Rashid, and W. Husain, “Data Mining in Healthcare – A Review,” Procedia Comput. Sci., vol. 72, pp. 306–313, 2015, doi: 10.1016/j.procs.2015.12.145.
H. Kaur and S. K. Wasan, “Empirical Study on Applications of Data Mining Techniques in Healthcare,” J. Comput. Sci., vol. 2, no. 2, pp. 194–200, Feb. 2006, doi: 10.3844/jcssp.2006.194.200.
S. M. Birjandi and S. H. Khasteh, “A survey on data mining techniques used in medicine,” J. Diabetes Metab. Disord., vol. 20, no. 2, pp. 2055–2071, Aug. 2021, doi: 10.1007/s40200-021-00884-2.
R. Bellazzi, F. Ferrazzi, and L. Sacchi, “Predictive data mining in clinical medicine: a focus on selected methods and applications,” WIREs Data Min. Knowl. Discov., vol. 1, no. 5, pp. 416–430, Sep. 2011, doi: 10.1002/widm.23.
J. A. Harding, M. Shahbaz, Srinivas, and A. Kusiak, “Data Mining in Manufacturing: A Review,” J. Manuf. Sci. Eng., vol. 128, no. 4, pp. 969–976, Nov. 2006, doi: 10.1115/1.2194554.
Y. Guo, W. Zhang, Q. Qin, K. Chen, and Y. Wei, “Intelligent manufacturing management system based on data mining in artificial intelligence energy-saving resources,” Soft Comput., vol. 27, no. 7, pp. 4061–4076, Apr. 2023, doi: 10.1007/s00500-021-06593-5.
S. Kamble, A. Desai, and P. Vartak, “Data mining and data warehousing for Supply Chain Management,” in 2015 International Con-ference on Communication, Information & Computing Technology (ICCICT), Jan. 2015, pp. 1–6. doi: 10.1109/ICCICT.2015.7045692.
A. Kusiak, “Data mining: manufacturing and service applications,” Int. J. Prod. Res., vol. 44, no. 18–19, pp. 4175–4191, Sep. 2006, doi: 10.1080/00207540600632216.
M. Er Kara, S. Ü. Oktay Fırat, and A. Ghadge, “A data mining-based framework for supply chain risk management,” Comput. Ind. Eng., vol. 139, p. 105570, Jan. 2020, doi: 10.1016/j.cie.2018.12.017.
M. Wu, K. Liu, and H. Yang, “Supply chain production and delivery scheduling based on data mining,” Cluster Comput., vol. 22, no. S4, pp. 8541–8552, Jul. 2019, doi: 10.1007/s10586-018-1894-8.
V. R. Rao, “A Framework for e-Government Data Mining Applications (eGDMA) for Effective Citizen Services-An Indian Per-spective,” Int. J. Comput. Sci. Inf. Technol. Res., vol. 2, no. 4, pp. 209–225, 2014.
M. M. Mostafa and A. A. El-Masry, “Citizens as consumers: Profiling e-government services’ users in Egypt via data mining tech-niques,” Int. J. Inf. Manage., vol. 33, no. 4, pp. 627–641, Aug. 2013, doi: 10.1016/j.ijinfomgt.2013.03.007.
S. De Cnudde and D. Martens, “Loyal to your city? A data mining analysis of a public service loyalty program,” Decis. Support Syst., vol. 73, pp. 74–84, May 2015, doi: 10.1016/j.dss.2015.03.004.
S. Ramos and Z. Vale, “Data mining techniques application in power distribution utilities,” in 2008 IEEE/PES Transmission and Distribution Conference and Exposition, Apr. 2008, pp. 1–8. doi: 10.1109/TDC.2008.4517229.
D. M. de Santana, S. R. Lourenço, and D. A. Cassiano, “Data mining approach for energy efficiency improvements in a utilities supply on a petrochemical plant,” Evol. Syst., vol. 14, no. 6, pp. 1071–1081, Dec. 2023, doi: 10.1007/s12530-023-09515-y.
R. R, E. S. Sankari, and Matheswaran.P, “Detection of non-technical loss in power utilities using data mining techniques,” Int. J. Innov. Res. Sci. Technol., vol. 1, no. 9, pp. 97–101, 2015.
S. Singh and A. Yassine, “Big Data Mining of Energy Time Series for Behavioral Analytics and Energy Consumption Forecasting,” Energies, vol. 11, no. 2, p. 452, Feb. 2018, doi: 10.3390/en11020452.
S. Ramos et al., “Data mining techniques for electricity customer characterization,” Procedia Comput. Sci., vol. 186, pp. 475–488, 2021, doi: 10.1016/j.procs.2021.04.168.
L. Zhu, M. Li, Z. Zhang, X. Du, and M. Guizani, “Big Data Mining of Users’ Energy Consumption Patterns in the Wireless Smart Grid,” IEEE Wirel. Commun., vol. 25, no. 1, pp. 84–89, Feb. 2018, doi: 10.1109/MWC.2018.1700157.
C. Costa and M. Y. Santos, “Improving cities sustainability through the use of data mining in a context of big city data,” in Interna-tional Conference of Data Mining and Knowledge Engineering, 2015. [Online]. Available: https://www.iaeng.org/publication/WCE2015/WCE2015_pp320-325.pdf
C. Cacciuttolo, V. Guzmán, P. Catriñir, E. Atencio, S. Komarizadehasl, and J. A. Lozano-Galant, “Low-Cost Sensors Technologies for Monitoring Sustainability and Safety Issues in Mining Activities: Advances, Gaps, and Future Directions in the Digitalization for Smart Mining,” Sensors, vol. 23, no. 15, p. 6846, Aug. 2023, doi: 10.3390/s23156846.
Y. Shi and U. Zhu, “Efficient Data Mining Method for Environmental Monitoring around Water Conservancy Projects,” Ekoloji Derg., vol. 28, no. 108, pp. 2447–2451, 2019.
K. Gibert, J. Izquierdo, M. Sànchez-Marrè, S. H. Hamilton, I. Rodríguez-Roda, and G. Holmes, “Which method to use? An as-sessment of data mining methods in Environmental Data Science,” Environ. Model. Softw., vol. 110, pp. 3–27, Dec. 2018, doi: 10.1016/j.envsoft.2018.09.021.
R. S. J. d. Baker, “Data Mining,” in International Encyclopedia of Education, Elsevier, 2010, pp. 112–118. doi: 10.1016/B978-0-08-044894-7.01318-X.
C. Romero and S. Ventura, “Data mining in education,” WIREs Data Min. Knowl. Discov., vol. 3, no. 1, pp. 12–27, Jan. 2013, doi: 10.1002/widm.1075.
A. Algarni, “Data Mining in Education,” Int. J. Adv. Comput. Sci. Appl., vol. 7, no. 6, 2016, doi: 10.14569/IJACSA.2016.070659.
K. R. Koedinger, S. D’Mello, E. A. McLaughlin, Z. A. Pardos, and C. P. Rosé, “Data mining and education,” WIREs Cogn. Sci., vol. 6, no. 4, pp. 333–353, Jul. 2015, doi: 10.1002/wcs.1350.
S. Agarwal, “Data Mining in Education: Data Classification and Decision Tree Approach,” Int. J. e-Education, e-Business, e-Management e-Learning, vol. 2, no. 2, pp. 140–144, 2012, doi: 10.7763/IJEEEE.2012.V2.97.
M. M. Arcinas, una S. Sajja, S. Asif, S. Gour, E. Okoronkwo, and M. Naved, “Role of data mining in education for improving students performance for social change,” Turkish J. Physiother. Rehabil., vol. 32, no. 3, pp. 6519–6526, 2021.
S. Suhirman, T. Herawan, H. Chiroma, and J. Mohamad Zain, “Data Mining for Education Decision Support: A Review,” Int. J. Emerg. Technol. Learn., vol. 9, no. 6, p. 4, Dec. 2014, doi: 10.3991/ijet.v9i6.3950.
M. Goyal and R. Vohra, “Applications of Data Mining in Higher Education,” Int. J. Comput. Sci. Issues, vol. 9, no. 2, pp. 113–120, 2012.
S. K. Malik, N. Prakash, and S. Rizvi, “Developing an university ontology in education domain using protégé for semantic web,” Int. J. Eng. Sci. Technol., vol. 2, no. 9, pp. 4673–4681, 2010.
A. Ameen, K. R. Khan, and B. P. Rani, “Construction of university ontology,” in 2012 World Congress on Information and Communication Technologies, Oct. 2012, pp. 39–44. doi: 10.1109/WICT.2012.6409047.
N. Malviya, N. Mishra, and S. Sahu, “Developing University Ontology using protégé OWL Tool: Process and Reasoning,” Int. J. Sci. Eng. Res., vol. 2, no. 9, 2011.
L. Zeng, T. Zhu, and X. Ding, “Study on Construction of University Course Ontology: Content, Method and Process,” in 2009 International Conference on Computational Intelligence and Software Engineering, Dec. 2009, pp. 1–4. doi: 10.1109/CISE.2009.5363158.
I. T. Ayorinde and O. A. Akinyele, “A Formal Activity Ontology for Postgraduate Admission Processes (A Case Study of University of Ibadan, Nigeria),” Ilorin J. Comput. Sci. Inf. Technol., vol. 1, no. 1, pp. 1–18, 2016.
G. T. Ayem, S. G. Thandekkattu, A. S. Nsang, and M. Fonkam, “Structural Causal Model Design and Causal Impact Analysis: A Case of SENSE-EGRA Dataset,” 2023, pp. 39–52. doi: 10.1007/978-981-99-3485-0_4.
G. T. Ayem, A. Ajibesin, A. Iorliam, and A. S. Nsang, “A mixed framework for causal impact analysis under confounding and se-lection biases: a focus on Egra dataset,” Int. J. Inf. Technol., Oct. 2023, doi: 10.1007/s41870-023-01490-6.
G. T. Ayem, A. S. Nsang, B. I. Igoche, and G. Naankang, “Design and Validation of Structural Causal Model: A focus on SENSE-EGRA Datasets,” Int. J. Adv. Sci. Comput. Eng., vol. 5, no. 3, pp. 257–268, Dec. 2023, doi: 10.62527/ijasce.5.3.177.
G. T. Ayem, O. Asilkan, and A. Iorliam, “Design and Validation of Structural Causal Model: A Focus on EGRA Dataset,” J. Comput. Theor. Appl., vol. 1, no. 2, pp. 86–103, Nov. 2023, doi: 10.33633/jcta.v1i2.9304.
S. Sheikholeslami, “Ablation Programming for Machine Learning,” KTH-Royal Institute of Technology, 2019.
S. Sheikholeslami, M. Meister, T. Wang, A. H. Payberah, V. Vlassov, and J. Dowling, “AutoAblation,” in Proceedings of the 1st Workshop on Machine Learning and Systems, Apr. 2021, pp. 55–61. doi: 10.1145/3437984.3458834.
M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?,’” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778.
J. Pearl, Causality: Models, Reasoning and Inference. University of California, Los Angeles: Cambridge University Press, 2009.
S. Greenland, J. Pearl, and J. M. Robins, “Causal diagrams for epidemiologic research.,” Epidemiology, vol. 10, no. 1, pp. 37–48, Jan. 1999, [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/9888278
J. Pearl, “Causal inference in statistics: An overview,” Stat. Surv., vol. 3, no. none, Jan. 2009, doi: 10.1214/09-SS057.
B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat., vol. 32, no. 2, Apr. 2004, doi: 10.1214/009053604000000067.
J. Budzianowski et al., “Machine learning model for predicting late recurrence of atrial fibrillation after catheter ablation,” Sci. Rep., vol. 13, no. 1, p. 15213, Sep. 2023, doi: 10.1038/s41598-023-42542-y.
C. K. Yeh, C. Y. Hsieh, A. S. Suggala, D. I. Inouye, and P. Ravikumar, “On the (In)fidelity and sensitivity of explanations,” Adv. Neural Inf. Process. Syst., vol. 32, no. NeurIPS, 2019, [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2019/file/a7471fdc77b3435276507cc8f2dc2569-Paper.pdf
A. Papenmeier, G. Englebienne, and C. Seifert, “How model accuracy and explanation fidelity influence user trust in AI,” in IJCAI 2019 Workshop on Explainable Artificial Intelligence, 2019.
R. Gaudel, L. Galárraga, J. Delaunay, L. Rozé, and V. Bhargava, “s-LIME: Reconciling Locality and Fidelity in Linear Explanations,” in Advances in Intelligent Data Analysis XX, 2022, pp. 102–114. doi: 10.1007/978-3-031-01333-1_9.
M. Velmurugan, C. Ouyang, C. Moreira, and R. Sindhgatta, “Developing a Fidelity Evaluation Approach for Interpretable Machine Learning.” Jun. 15, 2021. [Online]. Available: http://arxiv.org/abs/2106.08492
A. Angerschmid, J. Zhou, K. Theuermann, F. Chen, and A. Holzinger, “Fairness and Explanation in AI-Informed Decision Mak-ing,” Mach. Learn. Knowl. Extr., vol. 4, no. 2, pp. 556–579, Jun. 2022, doi: 10.3390/make4020026.
J. Dodge, Q. V. Liao, Y. Zhang, R. K. E. Bellamy, and C. Dugan, “Explaining models,” in Proceedings of the 24th International Conference on Intelligent User Interfaces, Mar. 2019, pp. 275–285. doi: 10.1145/3301275.3302310.
A. Balagopalan, H. Zhang, K. Hamidieh, T. Hartvigsen, F. Rudzicz, and M. Ghassemi, “The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations,” in 2022 ACM Conference on Fairness, Accountability, and Transparency, Jun. 2022, pp. 1194–1206. doi: 10.1145/3531146.3533179.
E. A. Murray, “What have ablation studies told us about the neural substrates of stimulus memory?,” Semin. Neurosci., vol. 8, no. 1, pp. 13–22, Feb. 1996, doi: 10.1006/smns.1996.0003.
B. Zhou, Y. Sun, D. Bau, and A. Torralba, “Revisiting the Importance of Individual Units in CNNs via Ablation.” Jun. 07, 2018. [Online]. Available: http://arxiv.org/abs/1806.02891
R. Meyes, M. Lu, C. W. de Puiseau, and T. Meisen, “Ablation Studies in Artificial Neural Networks.” Jan. 24, 2019. [Online]. Available: http://arxiv.org/abs/1901.08644
J. Pearl, “The seven tools of causal inference, with reflections on machine learning,” Commun. ACM, vol. 62, no. 3, pp. 54–60, Feb. 2019, doi: 10.1145/3241036.
C. Cinelli, D. Kumor, B. Chen, J. Pearl, and E. Bareinboim, “Sensitivity Analysis of Linear Structural Causal Models,” in Proceedings of the 36th International Conference on Machine Learning, 2019, pp. 1252–1261. [Online]. Available: https://proceedings.mlr.press/v97/cinelli19a.html
K. S. Gill, “Pearl, Judea and Mackenzie, Dana: The book of why: the new science of cause and effect (2018),” AI Soc., vol. 35, no. 3, pp. 767–768, Sep. 2020, doi: 10.1007/s00146-020-00971-7.
K. A. Markus, “Causal effects and counterfactual conditionals: contrasting Rubin, Lewis and Pearl,” Econ. Philos., vol. 37, no. 3, pp. 441–461, Nov. 2021, doi: 10.1017/S0266267120000437.
R. Briggs, “Interventionist counterfactuals,” Philos. Stud., vol. 160, no. 1, pp. 139–166, Aug. 2012, doi: 10.1007/s11098-012-9908-5.
A.-H. Karimi, B. Schölkopf, and I. Valera, “Algorithmic Recourse,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Mar. 2021, pp. 353–362. doi: 10.1145/3442188.3445899.
F. Wang and C. Rudin, “Falling Rule Lists,” in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, 2015, vol. 38, pp. 1013–1022. [Online]. Available: https://proceedings.mlr.press/v38/wang15a.html
J. Dai, S. Upadhyay, U. Aivodji, S. H. Bach, and H. Lakkaraju, “Fairness via Explanation Quality,” in Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, Jul. 2022, pp. 203–214. doi: 10.1145/3514094.3534159.
Y. Zhao, Y. Wang, and T. Derr, “Fairness and Explainability: Bridging the Gap towards Fair Model Explanations,” Proc. AAAI Conf. Artif. Intell., vol. 37, no. 9, pp. 11363–11371, Jun. 2023, doi: 10.1609/aaai.v37i9.26344.
J. Schoeffer and N. Kuehl, “Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems,” in Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, Oct. 2021, pp. 153–157. doi: 10.1145/3462204.3481742.
U. Bhatt et al., “Explainable machine learning in deployment,” in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Jan. 2020, pp. 648–657. doi: 10.1145/3351095.3375624.
S. Sharma, J. Henderson, and J. Ghosh, “CERTIFAI: A common framework to provide explanations and analyse the fairness and robustness of black-box models,” in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Feb. 2020, pp. 166–172. doi: 10.1145/3375627.3375812.
S. Tang et al., “Machine Learning–Enabled Multimodal Fusion of Intra-Atrial and Body Surface Signals in Prediction of Atrial Fibrillation Ablation Outcomes,” Circ. Arrhythmia Electrophysiol., vol. 15, no. 8, Aug. 2022, doi: 10.1161/CIRCEP.122.010850.
J. C. Miller, Laser Ablation Principles and Applications, vol. 28. Berlin, Heidelberg: Springer Berlin Heidelberg, 1994. doi: 10.1007/978-3-642-78720-1.
G. T. Ayem, O. Asilkan, A. Iorliam, R. Ibrahim, and S. George, “Causal Inference Estimates with Backdoor Adjustment Condition vs. the Unconfoundedness Assumption: A Comparative Analysis Study of the Structural Causal Model and the Potential Outcome Frameworks,” ESP J. Eng. Technol. Adv., 2023, doi: 10.56472/25832646/JETA-V3I8P101.
R. U. Onyekwelu and A. J. Obikeze, “E-Administration And Service Delivery: A Study Of Joint Admissions And Matriculation Board (JAMB) Nigeria,” Int. J. Innov. Dev. Policy Stud., 2023, [Online]. Available: http://seahipaj.org/journals-ci/mar-2023/IJIDPS/full/IJIDPS-M-5-2023.pdf
A. Ankan, I. M. N. Wortel, and J. Textor, “Testing Graphical Causal Models Using the R Package ‘dagitty,’” Curr. Protoc., vol. 1, no. 2, Feb. 2021, doi: 10.1002/cpz1.45.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Bern Igoche Igoche
This work is licensed under a Creative Commons Attribution 4.0 International License.
