Design and Validation of Structural Causal Model: A Focus on EGRA Dataset

Gabriel Terna Ayem; Ozcan Asilkan; Aamo Iorliam

doi:10.33633/jcta.v1i2.9304

Design and Validation of Structural Causal Model: A Focus on EGRA Dataset

Gabriel Terna Ayem, Ozcan Asilkan, Aamo Iorliam

Abstract

Designing and validating structural causal model (SCM) correctness from a dataset whose background knowledge is obtained from a research process is not a common phenomenon. Studies have shown that in many critical areas, such as healthcare and education, researchers develop models from direct acyclic graphs (DAG), a component of an SCM, without testing them. This phenomenon is worrisome and is bound to cast a shadow on the inference estimates that may arise from such models. In this study, we have designed a novel application-based SCM for the first time using the background knowledge obtained from the Early Grade Reading Assessment (EGRA) program called the Strengthen Education in Northeast Nigeria (SENSE-EGRA), which is an educational intervention program of the American University of Nigeria (AUN), Yola, on the letter identification subtask. This project was sponsored by the United States Agency for International Development (USAID). We employed the conditional independence test (CIT) criteria for the validation of the SCM’s correctness, and the results show a near-perfect SCM.

Keywords

Structural causal models; Causal validation; Conditional independent test; Observational datasets; EGRA

Full Text:

PDF

References

J. F. Box, “RA Fisher, the Life of a Scientist,” Revue Philosophique de la France Et de l, vol. 170, no. 4, 1980.

K. Benson and A. J. Hartz, “A Comparison of Observational Studies and Randomized, Controlled Trials,” N. Engl. J. Med., vol. 342, no. 25, pp. 1878–1886, Jun. 2000, doi: 10.1056/NEJM200006223422506.

S. L. Silverman, “From Randomized Controlled Trials to Observational Studies,” Am. J. Med., vol. 122, no. 2, pp. 114–120, Feb. 2009, doi: 10.1016/j.amjmed.2008.09.030.

P. W. G. Tennant et al., “Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations,” Int. J. Epidemiol., vol. 50, no. 2, pp. 620–632, May 2021, doi: 10.1093/ije/dyaa213.

S. Greenland, J. Pearl, and J. M. Robins, “Causal diagrams for epidemiologic research,” Epidemiology, pp. 37-48, 1999, doi: 10.1097/00001648-199901000-00008.

J. Pearl, Causality: Models, Reasoning and Inference, 2nd ed. University of California, Los Angeles: Cambridge University Press, 2009.

J. Tian and J. Pearl, “Department of Statistics Papers A General Identification Condition for Causal Effects,” UCLA Dep. Stat. Pap., 2002.

G. T. Ayem, S. G. Thandekkattu, A. S. Nsang, and M. Fonkam, “Structural Causal Model Design and Causal Impact Analysis: A Case of SENSE-EGRA Dataset,” 2023, pp. 39–52. doi: 10.1007/978-981-99-3485-0_4.

R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection." International Joint Conference on Artifical Intelligence, (IJCAI), pp. 1137-1145, 1995.

S. Borra and A. Di Ciaccio, “Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods,” Comput. Stat. Data Anal., vol. 54, no. 12, pp. 2976–2989, Dec. 2010, doi: 10.1016/j.csda.2010.03.004.

A. Shojaie, and E. B. Fox, “Granger causality: A review and recent advances,” Annual Review of Statistics and Its Application, vol. 9, pp. 289-319, 2022, doi: 10.1146/annurev-statistics-040120-010930.

S. L. Bressler, and A. K. Seth, “Wiener–Granger causality: a well-established methodology,” Neuroimage, vol. 58, no. 2, pp. 323-329, 2011, doi: 10.1016/j.neuroimage.2010.02.059.

E. Bareinboim, and J. Pearl, "Controlling selection bias in causal inference." Artificial intelligence and Statistics.PMLR, pp. 100-108, 2012.

E. Bareinboim, J. Tian, and J. Pearl, “Recovering from Selection Bias in Causal and Statistical Inference (Supplemental Material),” 2014.

G. T. Ayem, A. Ajibesin, A. Iorliam, and A. S. Nsang, “A mixed framework for causal impact analysis under confounding and selection biases: a focus on Egra dataset,” International Journal of Information Technology, 2023, doi:10.1007/s41870-023-01490-6

B. Piper, S. Dryden-Peterson, V. Chopra, C. Reddick, and A. Oyanga, “Are refugee children learning? Early grade literacy in a refugee camp in Kenya,” Journal on Education in Emergencies, vol.5 (2): pp.71-107 2020, doi:10.33682/f1wr-yk6y.

S. Taylor, J. Cilliers, C. Prinsloo, B. Fleisch, and V. Reddy, “The Early Grade Reading Study: Impact evaluation after two years of interventions,” EGRS Evaluation Report, 2017.

B. Fleisch, V. Schöer, G. Roberts, and A. Thornton, “System-wide improvement of early-grade mathematics: New evidence from the Gauteng Primary Language and Mathematics Strategy,” International Journal of Educational Development, vol. 49, pp. 157-174, 2016, doi: 10.1016/j.ijedudev.2016.02.0.

L. O. Costa, and M. Carnoy, “The effectiveness of an early-grade literacy intervention on the cognitive achievement of Brazilian students,” Educational Evaluation and Policy Analysis, vol. 37, no. 4, pp. 567-590, 2015, doi: 10.3102/0162373715571437.

B. Piper, and M. Korda, “EGRA Plus: Liberia. Program Evaluation Report,” RTI International, 2011.

M. Davidson, M. Korda, and O. W. Collins, “Teachers’ use of EGRA for continuous assessment: the case of EGRA Plus: Liberia,” The Early Grade Reading Assessment, pp. 113, 2011.

M. Davidson and J. Hobbs, “Delivering reading intervention to the poorest children: The case of Liberia and EGRA-Plus, a primary grade reading assessment and intervention,” Int. J. Educ. Dev., vol. 33, no. 3, pp. 283–293, May 2013, doi: 10.1016/j.ijedudev.2012.09.005.

M. M. de Oca, J. Hill, L. Aber, C. T. Dolan, and K. Gjicali, “The Impact of Attending a Remedial Support Program on Syrian Children's Reading Skills: Using BART for Causal Inference,” arXiv preprint arXiv:2208.13906, 2022.

R. Guo, L. Cheng, J. Li, P. R. Hahn, and H. Liu, “A Survey of Learning Causality with Data,” ACM Comput. Surv., vol. 53, no. 4, pp. 1–37, Jul. 2021, doi: 10.1145/3397269.

C. Hitchcock, and M. Rédei, “Reichenbach’s common cause principle,” 2020, online at.< Reichenbach’s Common Cause Principle (Stanford Encyclopedia of Philosophy>.

J. Pearl, Causality. Cambridge University Press, 2009. doi: 10.1017/CBO9780511803161.

J. Peters, D. Janzing, and B. Scholkopf, Elements of causal inference. London, England: MIT Press, 2017.

J. Neyman, “Sur les applications de la theorie des probabilities aux experiences agricoles: essai des principes (Masters Thesis); Justification of applications of the calculus of probabilities to the solutions of certain questions in agricultural experimentation. Excerpts English translation (Reprinted),” Stat Sci, vol. 5, pp. 463-472, 1923.

D. B. Rubin, “Estimating causal effects of treatments in randomized and nonrandomized studies,” Journal of Educational Psychology, vol. 66, no. 5, pp. 688, 1974.

V. Karwa, A. B. Slavkovi?, and E. T. Donnell, “Causal inference in transportation safety studies: Comparison of potential outcomes and causal diagrams,” The Annals of Applied Statistics, vol 5. 2011, doi: :10.1214/10-AOAS440.

C. R. Lesko, A. L. Buchanan, D. Westreich, J. K. Edwards, M. G. Hudgens, and S. R. Cole, “Generalizing Study Results,” Epidemiology, vol. 28, no. 4, pp. 553–561, Jul. 2017, doi: 10.1097/EDE.0000000000000664..

G. W. Imbens, “Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics,” Journal of Economic Literature, vol. 58, no. 4, pp. 1129-1179, 2020, doi: 10.1257/jel.20191597.

P. Spirtes, C. Glymour, and R. Scheines, “Discovery Algorithms for Causally Sufficient Structures,” 1993, pp. 103–162. doi: 10.1007/978-1-4612-2748-9_5.

S. Lauritzen, “Causal Inference from Graphical Models,” 2000. doi: 10.1201/9781420035988.ch2.

F. Eberhardt, “Introduction to the foundations of causal discovery,” International Journal of Data Science and Analytics, vol. 3, no. 2, pp. 81-91, 2017, doi: 10.1007/s41060-016-0038-6.

J. Y. Halpern, "The Book of Why, Judea Pearl, Basic Books (2018)," Elsevier, 2019.

B. Neal, “Introduction to causal inference from a machine learning perspective,” Course Lecture Notes (draft), 2020.

F. Elwert, “Graphical Causal Models,” 2013, pp. 245–273. doi: 10.1007/978-94-007-6094-3_13..

A. R. Nogueira, A. Pugnana, S. Ruggieri, D. Pedreschi, and J. Gama, “Methods and tools for causal discovery and causal inference,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, pp. e1449, 2022, doi: 10.1002/widm.1449.

L. Yao, Z. Chu, S. Li, Y. Li, J. Gao, and A. Zhang, “A survey on causal inference,” ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 15, no. 5, pp. 1-46, 2021, doi:10.1145/3444944.

C. Glymour, K. Zhang, and P. Spirtes, “Review of causal discovery methods based on graphical models,” Frontiers in genetics, vol. 10, pp. 524, 2019, doi: 10.3389/fgene.2019.00524.

J. Pearl, Probabilistic reasoning in intelligent systems: networks of plausible inference: Morgan Kaufmann, 1988.

L. Gultchin, M. Kusner, V. Kanade, and R. Silva, "Differentiable causal backdoor discovery." in Proceedings of the Twenty Third Inter-national Conference on Artificial Intelligence and Statistics, PMLR, pp. 3970-3979, 2020, doi:108:3970-3979.

J. Correa, and E. Bareinboim, "Causal effect identification by adjustment under confounding and selection biases."in Thirty-First Conference on Artificial intelligence, San Francisco, CA, 2017.

J. Pearl, and D. Mackenzie, The book of why: the new science of cause and effect: Basic books, 2018.

J. Pearl, “Theoretical impediments to machine learning with seven sparks from the causal revolution,” in Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining - WSDM, 2018, doi: 10.1145/3159652.3176182.

T. S. Richardson, and J. M. Robins, "Single world intervention graphs: a primer.", 2013.

J. Zhang, and P. Spirtes, “Intervention, determinism, and the causal minimality condition,” Synthese, vol. 182, no. 3, pp. 335-347, 2011, doi: 10.1007/s11229-010-9751-1.

A. Hauser and P. Bühlmann, “Characterization and greedy learning of interventional markov equivalence classes of directed acyclic graphs,” J. Mach. Learn. Res., vol. 13, pp. 2409–2464, 2012.

A. Hauser, and P. Bühlmann, “Jointly interventional and observational data: estimation of interventional Markov equivalence classes of directed acyclic graphs,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 77, no. 1, pp. 291-318, 2015, doi:10.1111/rssb.12071.

R. Mayrhofer, and M. R. Waldmann, “Sufficiency and necessity assumptions in causal structure induction,” Cognitive science, vol. 40, no. 8, pp. 2137-2150, 2016, doi:10.1111/cogs.12318.

J. Zhang, and W. Mayer, “Weakening faithfulness: some heuristic causal discovery algorithms,” International Journal of data science and Analytics, vol. 3, no. 2, pp. 93-104, 2017, doi: 10.1007/s41060-016-0033-y.

J. Zhang, and P. L. Spirtes, “Strong faithfulness and uniform consistency in causal inference,” in Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence (UAI2003), 2012, doi:10.48550/arXiv.1212.2506.

American University of Nigeria (AUN), "SENSE project evaluation: reading proficiency assessment, results of the hausa early grade reading assessment (EGRA) in Adamawa And Gombe," a report, 2021.online at.< SENSE-EGRA_Project/SENSE_EGRA Evaluation report_20211026.pdf at main · Sadaju-Codes/SENSE-EGRA_Project (github.com)>.

I. Shpitser, T. VanderWeele, and J. M. Robins, “On the validity of covariate adjustment for estimating causal effects,” Proc. 26th Conf. Uncertain. Artif. Intell. UAI 2010, pp. 527–536, 2010.

A. Ankan, I. M. Wortel, and J. Textor, “Testing graphical causal models using the R package “dagitty”,” Current Protocols, vol. 1, no. 2, pp. e45, 2021, doi:10.1002/cpz1.45

F. Thoemmes, Y. Rosseel, and J. Textor, “Local fit evaluation of structural equation models using graphical criteria,” Psychological methods, vol. 23, no. 1, pp. 27, 2018, doi:10.1037/met0000147.

J. Pearl and T. S. Verma, “A theory of inferred causation,” 1995, pp. 789–811. doi: 10.1016/S0049-237X(06)80074-1..

J. Pearl and E. Bareinboim, “Transportability of Causal and Statistical Relations: A Formal Approach,” Proc. AAAI Conf. Artif. Intell., vol. 25, no. 1, pp. 247–254, Aug. 2011, doi: 10.1609/aaai.v25i1.7861.

J. Pearl, “Causal inference in statistics: An overview,” Statistics surveys, vol. 3, pp. 96-146, 2009, doi: 10.1214/09-SS057.

DOI: https://doi.org/10.33633/jcta.v1i2.9304

Article Metrics

Abstract view : 54 times
PDF - 59 times

Refbacks

There are currently no refbacks.

Journal of Computing Theories and Applications published by Dian Nuswantoro University, Semarang, Indonesia collaborates with the Intelligent System (IntSys) Research and is supported by Future Techno Science Foundation, Indonesia.

Indexed by: