A Five-Step Multi-Dimensional Learnability Validation Protocol for Meningitis Clinical Risk Assessment
DOI:
https://doi.org/10.62411/jcta.18090Keywords:
Brier score decomposition, Clinical decision support, Decision curve analysis, Explainable artificial intelligence, Gradient boosting, Meningitis classification, Model calibrationAbstract
Clinical machine learning is routinely judged on discrimination alone. A model that separates classes well is reported as successful, even when its probabilities are miscalibrated, its explanations point at variables with no measured association with the outcome, and nobody has checked whether the target can be learned from the available features at all. This study proposes a five-step multidimensional learnability validation protocol that tests a prediction target on five axes: feature-target dependence before modelling, calibration through Brier score decomposition, clinical utility through decision curve analysis, interpretability through SHapley Additive exPlanations, and agreement between fitted coefficient signs and the directions reported in the clinical literature. The protocol is the contribution; no new learning algorithm is proposed. Five established classifiers, Logistic Regression, Random Forest, LightGBM, CatBoost, and XGBoost, are used as instruments on a public synthetic meningitis dataset of 1,200 records with 10 clinical predictors and three prediction targets. Under repeated stratified cross-validation with five folds and five repeats, the four tree-based models reached mean areas under the curve of 0.969 to 0.971 and mean Matthews correlation coefficients of 0.780 to 0.793 for aetiological diagnosis, and 0.969 to 0.972 and 0.791 to 0.802 for clinical risk level; both targets passed all five validation steps. The mortality target failed all five: the strongest feature-target correlation was 0.067 and the strongest mutual information 0.013, the Brier score of the Platt-scaled reference model (0.0949) did not fall below the uncertainty baseline (0.0944), no threshold between 0.05 and 0.50 gave net benefit above both default strategies, the leading SHAP attribution fell on a variable with no measured association, and only two of five literature-referenced coefficient signs agreed. Judged only by whether mean discrimination exceeded chance, mortality could have appeared marginally viable. The protocol is demonstrated, not externally validated, and a target that passes is eligible for further validation rather than ready for clinical use.References
J. R. Zunt et al., “Global, regional, and national burden of meningitis, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016,” Lancet Neurol., vol. 17, no. 12, pp. 1061–1082, Dec. 2018, doi: 10.1016/S1474-4422(18)30387-9.
D. van de Beek, J. de Gans, L. Spanjaard, M. Weisfelt, J. B. Reitsma, and M. Vermeulen, “Clinical Features and Prognostic Factors in Adults with Bacterial Meningitis,” N. Engl. J. Med., vol. 351, no. 18, pp. 1849–1859, Oct. 2004, doi: 10.1056/NEJMoa040845.
B. Dobardžić, A. Alibašić, N. Milošević, B. Mališić, and M. Vukotić, “Forecasting Meningitis with Machine Learning: An Advanced Classification Model Analysis,” in IFMBE Proceedings, vol. 93, 2024, pp. 725–731. doi: 10.1007/978-3-031-49062-0_76.
V. M. Lelis, E. Guzmán, and M.-V. Belmonte, “Non-Invasive Meningitis Diagnosis Using Decision Trees,” IEEE Access, vol. 8, pp. 18394–18407, 2020, doi: 10.1109/ACCESS.2020.2966397.
E. Guzmán, M. Belmonte, and V. M. Lelis, “Ensemble methods for meningitis aetiology diagnosis,” Expert Syst., vol. 39, no. 8, Sep. 2022, doi: 10.1111/exsy.12996.
S. Sathiya Priya, M. Amanullah, and T. Kujani, “Enhanced Meningitis Type Classification Using Ensemble Learning: A Hybrid LightGBM Model with Advanced Preprocessing,” in 2025 9th International Conference on Inventive Systems and Control (ICISC), Aug. 2025, pp. 763–768. doi: 10.1109/ICISC65841.2025.11188314.
S. Sathiya Priya, M. Amanullah, and T. Kujani, “A Hybrid Model for Meningitis Diagnostic Accuracy with Ensemble Learning: Combining Logistic Regression and L-BFGS,” in 2025 Second International Conference on Cognitive Robotics and Intelligent Systems (ICC - ROBINS), Jun. 2025, pp. 290–295. doi: 10.1109/ICC-ROBINS64345.2025.11086348.
S. Sathiya Priya, M. Amanullah, and T. Kujani, “Enhancing Meningitis Type Diagnosis through a Novel Hybrid Ensemble of Bagging and Gradient Boosting,” in 2024 Second International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI), Aug. 2024, pp. 1260–1264. doi: 10.1109/ICoICI62503.2024.10696600.
S. Sathiya Priya, M. Amanullah, and P. Devi, “High-Performance Meningitis Etiology Prediction: Leveraging CSF Biomarkers Via a Stacked Ensemble of Random Forest and Artificial Neural Networks,” in 2025 9th International Conference on Inventive Systems and Control (ICISC), Aug. 2025, pp. 1226–1232. doi: 10.1109/ICISC65841.2025.11188299.
S. Sathiya Priya, M. Amanullah, and P. Harini, “Fusion-based CNN-SVM Hybrid Model for Efficient Meningitis Identification using CSF Dataset,” in 2025 3rd International Conference on Sustainable Computing and Data Communication Systems (ICSCDS), Aug. 2025, pp. 1115–1119. doi: 10.1109/ICSCDS65426.2025.11167675.
A. Shabana, P. Kavitha, and S. Kamalakkannan, “Detection of meningitis disease using Belief Bidirectional Neural Network and Informative Ant Colony Optimization techniques,” Indian J. Eng., vol. 22, no. 58, pp. 1–17, Aug. 2025, doi: 10.54905/disssi.v22i58.e12ije1701.
A.-F. A. Mentis et al., “Artificial Intelligence in Differential Diagnostics of Meningitis: A Nationwide Study,” Diagnostics, vol. 11, no. 4, p. 602, Mar. 2021, doi: 10.3390/diagnostics11040602.
A. Spanos, F. E. Harrell, and D. T. Durack, “Differential diagnosis of acute meningitis: an analysis of the predictive value of initial observations,” JAMA, vol. 262, no. 19, pp. 2700–2707, Nov. 1989, doi: 10.1001/jama.1989.03430190084036.
L. E. Nigrovic et al., “Clinical Prediction Rule for Identifying Children With Cerebrospinal Fluid Pleocytosis at Very Low Risk of Bacterial Meningitis,” JAMA, vol. 297, no. 1, pp. 52–60, Jan. 2007, doi: 10.1001/jama.297.1.52.
J. Zhou, J. Xing, X. Cheng, F. Jin, and Y. Huang, “Development and validation of a predictive model for 30-day mortality in adult bacterial meningitis: a retrospective cohort study,” Front. Med., vol. 12, Nov. 2025, doi: 10.3389/fmed.2025.1692631.
Y. Luo et al., “Construction and validation of a predictive model for meningoencephalitis in pediatric scrub typhus based on machine learning algorithms,” Emerg. Microbes Infect., vol. 14, no. 1, Dec. 2025, doi: 10.1080/22221751.2025.2469651.
B. Zhou, F. Liang, Y. Huang, S. Zhang, Z. Zhuo, and C. Chen, “A predictive model for central nervous system infections in children based on machine learning and clinical diagnostic features,” Front. Pediatr., vol. 14, pp. 1–13, Jul. 2026, doi: 10.3389/fped.2026.1861527.
M. Yu, Y. Fang, and X. Yang, “Development and internal validation of a prognostic nomogram for poor outcomes in neonatal bacterial meningitis,” Transl. Pediatr., vol. 15, no. 5, p. 177, May 2026, doi: 10.21037/tp-2026-0206.
Y. Chen et al., “Development of a machine learning-based predictive model for long-term adverse outcomes in neonatal bacterial meningitis,” J. Pediatr. (Rio J.), vol. 102, no. 1, p. 101472, Jan. 2026, doi: 10.1016/j.jped.2025.101472.
D. Van Dong et al., “Causative pathogens and predictors of unfavourable outcomes in central nervous system infections in resource-limited settings,” Int. J. Infect. Dis., vol. 159, p. 108002, Oct. 2025, doi: 10.1016/j.ijid.2025.108002.
J. Milburn, R. Suresh, R. Doyle, and J. N. Jarvis, “The diagnosis of central nervous system infections in resource-limited settings and the use of novel and molecular diagnostic platforms to improve diagnosis,” Expert Rev. Mol. Diagn., vol. 24, no. 3, pp. 219–230, Mar. 2024, doi: 10.1080/14737159.2024.2317414.
G. Lu et al., “Prediction model of central nervous system infections in patients with severe traumatic brain injury after craniotomy,” J. Hosp. Infect., vol. 136, pp. 90–99, Jun. 2023, doi: 10.1016/j.jhin.2023.04.004.
R. Elshawi, M. H. Al-Mallah, and S. Sakr, “On the interpretability of machine learning-based model for predicting hypertension,” BMC Med. Inform. Decis. Mak., vol. 19, no. 1, p. 146, Dec. 2019, doi: 10.1186/s12911-019-0874-0.
G. Stiglic, P. Kocbek, N. Fijacko, M. Zitnik, K. Verbert, and L. Cilar, “Interpretability of machine learning‐based prediction models in healthcare,” WIREs Data Min. Knowl. Discov., vol. 10, no. 5, Sep. 2020, doi: 10.1002/widm.1379.
B. Van Calster, D. J. McLernon, M. van Smeden, L. Wynants, and E. W. Steyerberg, “Calibration: the Achilles heel of predictive analytics,” BMC Med., vol. 17, no. 1, p. 230, Dec. 2019, doi: 10.1186/s12916-019-1466-7.
S. M. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” in NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems, Nov. 2017, pp. 4768–4777. [Online]. Available: https://dl.acm.org/doi/10.5555/3295222.3295230
A. Messai et al., “Transparent AI Models for Meningococcal Meningitis Diagnosis: Evaluating Interpretability and Performance Metrics,” in 2024 IEEE 12th International Conference on Intelligent Systems (IS), Aug. 2024, pp. 1–8. doi: 10.1109/IS61756.2024.10705255.
C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” Nat. Mach. Intell., vol. 1, no. 5, pp. 206–215, May 2019, doi: 10.1038/s42256-019-0048-x.
G. Y. Gombolay et al., “Effects of explainable artificial intelligence in neurology decision support,” Ann. Clin. Transl. Neurol., vol. 11, no. 5, pp. 1224–1235, May 2024, doi: 10.1002/acn3.52036.
L. Bergomi et al., “Which explanations do clinicians prefer? A comparative evaluation of XAI understandability and actionability in predicting the need for hospitalization,” BMC Med. Inform. Decis. Mak., vol. 25, no. 1, p. 269, Jul. 2025, doi: 10.1186/s12911-025-03045-0.
A. H. Murphy, “A New Vector Partition of the Probability Score,” J. Appl. Meteorol., vol. 12, no. 4, pp. 595–600, Jun. 1973, doi: 10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2.
A. J. Vickers and E. B. Elkin, “Decision Curve Analysis: A Novel Method for Evaluating Prediction Models,” Med. Decis. Mak., vol. 26, no. 6, pp. 565–574, Nov. 2006, doi: 10.1177/0272989X06295361.
N. S. Groeneveld et al., “Diagnostic prediction models for bacterial meningitis in children with a suspected central nervous system infection: a systematic review and prospective validation study,” BMJ Open, vol. 14, no. 8, p. e081172, Aug. 2024, doi: 10.1136/bmjopen-2023-081172.
G. S. Collins, J. B. Reitsma, D. G. Altman, and K. G. M. Moons, “Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement,” BMJ, vol. 350, p. g7594, Jan. 2015, doi: 10.1136/bmj.g7594.
E. W. Steyerberg et al., “Assessing the performance of prediction models: a framework for traditional and novel measures,” Epidemiology, vol. 21, no. 1, pp. 128–138, Jan. 2010, doi: 10.1097/EDE.0b013e3181c30fb2.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, Jun. 2002, doi: 10.1613/jair.953.
R. Blagus and L. Lusa, “SMOTE for high-dimensional class-imbalanced data,” BMC Bioinformatics, vol. 14, no. 1, p. 106, Dec. 2013, doi: 10.1186/1471-2105-14-106.
A. Bastug, B. O. Ozbay, E. Oksuz, and O. Karabay, “Systematic Review of Artificial Intelligence Implementations in Central Nervous System Infections,” Mediterr. J. Infect. Microbes Antimicrob., vol. 12, Jul. 2023, doi: 10.4274/mjima.galenos.2023.2023.13.
A. Hameed Mousa, E. M. Thabit A. Alsaadi, M. A. Mohammed, H. Mezher Merdas, and S. Dakhil Khalaf, “Evaluating the Impact of SMOTE and SHAP on Machine Learning Classifiers: Enhancing Predictive Performance through Imbalance Mitigation and Interpretability,” Adv. Artif. Intell. Mach. Learn., vol. 06, no. 02, pp. 5339–5357, 2026, doi: 10.54364/AAIML.2026.62296.
C. Yang, E. A. Fridgeirsson, J. A. Kors, J. M. Reps, and P. R. Rijnbeek, “Impact of random oversampling and random undersampling on the performance of prediction models developed using observational health data,” J. Big Data, vol. 11, no. 1, p. 7, Jan. 2024, doi: 10.1186/s40537-023-00857-7.
M. Sobhan and A. M. Mondal, “Evaluating SHAP’s Robustness in Precision Medicine: Effect of Filtering and Normalization,” in 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Dec. 2023, pp. 3157–3164. doi: 10.1109/BIBM58861.2023.10385704.
A. J. Vickers and F. Holland, “Decision curve analysis to evaluate the clinical benefit of prediction models,” Spine J., vol. 21, no. 10, pp. 1643–1648, Oct. 2021, doi: 10.1016/j.spinee.2021.02.024.
Y. Wu and A. Wang, “Calibrated and Explainable CIN2+ Risk Stratification Using Routine Clinical Data: Development and External Validation,” Int. J. Womens. Health, vol. 18, pp. 1–14, Jun. 2026, doi: 10.2147/IJWH.S614964.
H. Johnston, N. Nair, and D. Du, “Estimating Calibrated Risks Using Focal Loss and Gradient-Boosted Trees for Clinical Risk Prediction,” Electronics, vol. 14, no. 9, p. 1838, Apr. 2025, doi: 10.3390/electronics14091838.
E. Winarno, I. M. Nur, A. Karim, S. Amri, I. E. Wirdati, and P. W. Adi, “An Explainable Multimodal Framework for Chest X-Ray Alert Classification Using Radiology Reports and Images,” J. Comput. Theor. Appl., vol. 3, no. 4, pp. 647–666, May 2026, doi: 10.62411/jcta.16023.
D. B. Firmawan and B. R. P. Darnoto, “Cross-Domain Faithfulness Evaluation of SHAP and Attention-Based Explanations in Transformer NLP Models,” J. Comput. Theor. Appl., vol. 4, no. 1, pp. 146–163, Jul. 2026, doi: 10.62411/jcta.16258.
C. Prakash, A. Sisodia, and M. Lind, “A Systematic Review of Agentic AI in Healthcare: An Evidence-Informed Seven-Principle Framework,” J. Comput. Theor. Appl., vol. 4, no. 1, pp. 43–73, Jun. 2026, doi: 10.62411/jcta.16171.
G. Ke et al., “LightGBM: A Highly Efficient Gradient Boosting Decision Tree,” Adv. Neural Inf. Process. Syst., vol. 30, 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html
L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “CatBoost: unbiased boosting with categorical features,” in Advances in Neural Information Processing Systems 31 (NeurIPS 2018), Dec. 2018. [Online]. Available: http://arxiv.org/abs/1706.09516
T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, pp. 785–794. doi: 10.1145/2939672.2939785.
L. Breiman, “Random Forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, Oct. 2001, doi: 10.1023/A:1010933404324.
L. S. Shapley, “A value for n-person games,” in Contributions to the Theory of Games (AM-28), Volume II, vol. 2. Princeton, NJ, USA: Princeton University Press, 1953, pp. 307–317. doi: 10.1515/9781400881970-018.
S. Kaufman, S. Rosset, C. Perlich, and O. Stitelman, “Leakage in data mining: formulation, detection, and avoidance,” ACM Trans. Knowl. Discov. Data, vol. 6, no. 4, Art. no. 15, Dec. 2012, doi: 10.1145/2382577.2382579.
S. F. Alnomasy, B. S. Alotaibi, A. H. Mujamammi, E. A. Hassan, and M. E. Ali, “Microbial aspects and potential markers for differentiation between bacterial and viral meningitis among adult patients,” PLoS One, vol. 16, no. 6, p. e0251518, Jun. 2021, doi: 10.1371/journal.pone.0251518.
L. U. Gerdes, P. E. Jørgensen, E. Nexø, and P. Wang, “C-reactive protein and bacterial meningitis: a meta-analysis,” Scand. J. Clin. Lab. Invest., vol. 58, no. 5, pp. 383–393, Jan. 1998, doi: 10.1080/00365519850186364.
S. Singh, K. Mahto, A. Kumar, P. Kumar, S. Kumar, and M. K. Prasad, “Diagnostic Test Accuracy of Serum and Cerebrospinal Fluid C-Reactive Protein in Bacterial Meningitis: A Systematic Review and Meta-Analysis,” Infect. Chemother., vol. 57, no. 2, p. 248, 2025, doi: 10.3947/ic.2024.0139.
M. W. Bijlsma et al., “Community-acquired bacterial meningitis in adults in the Netherlands, 2006–14: a prospective cohort study,” Lancet Infect. Dis., vol. 16, no. 3, pp. 339–347, Mar. 2016, doi: 10.1016/S1473-3099(15)00430-2.
S. Tubiana et al., “Community-acquired bacterial meningitis in adults: in-hospital prognosis, long-term disability and determinants of outcome in a multicentre prospective cohort,” Clin. Microbiol. Infect., vol. 26, no. 9, pp. 1192–1200, Sep. 2020, doi: 10.1016/j.cmi.2019.12.020.
N. Chekrouni et al., “Characteristics and prognostic factors of bacterial meningitis in the intensive care unit: a prospective nationwide cohort study,” Ann. Intensive Care, vol. 13, no. 1, p. 124, 2023, doi: 10.1186/s13613-023-01218-6.
ChanTest, “Meningitis Classification,” Kaggle, 2025. https://www.kaggle.com/datasets/chantest/meningitis-classification
N. Hidayat, L. Afuan, and H. R. Jannah, “Prescriptive Learning Analytics for Student Dropout: Integrating Temporal Velocity and Counterfactual Explanations in Longitudinal Data,” J. Comput. Theor. Appl., vol. 3, no. 4, pp. 627–646, May 2026, doi: 10.62411/jcta.15920.
M. G. S. P. Kusuma et al., “Integrating Quantum, Deep, and Classic Features with Attention-Guided AdaBoost for Medical Risk Prediction,” J. Comput. Theor. Appl., vol. 3, no. 2, pp. 171–189, Oct. 2025, doi: 10.62411/jcta.14873.
J. C. Platt, “Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods,” in Advances in Large-Margin Classifiers, The MIT Press, 2000, pp. 61–74. doi: 10.7551/mitpress/1113.003.0008.
F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011. [Online]. Available: https://jmlr.org/papers/v12/pedregosa11a.html
D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, p. 6, Dec. 2020, doi: 10.1186/s12864-019-6413-7.
A. Niculescu-Mizil and R. Caruana, “Predicting good probabilities with supervised learning,” in Proceedings of the 22nd international conference on Machine learning - ICML ’05, 2005, pp. 625–632. doi: 10.1145/1102351.1102430.
O. A. Abioye and M. E. Irhebhude, “Big Data-Driven Health Risk Stratification: A Health Index-Based Approach Using Feature Importance and PySpark,” J. Comput. Theor. Appl., vol. 2, no. 4, pp. 456–469, Mar. 2025, doi: 10.62411/jcta.12327.
A. J. Yadav and S. Mohammed Aslam, “Platelet indices as prognostic markers in the critically ill patient,” Ital. J. Med., vol. 15, no. 1, pp. 34–40, Dec. 2020, doi: 10.4081/itjm.2020.1388.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Fathimah Al-Ma'shumah, Yasi Dani, Yani Triyani, Dani Suandi

This work is licensed under a Creative Commons Attribution 4.0 International License.














