Causal Inference in High-Dimensional Settings: Machine Learning Integration and Recent Advances

Abstract

Causal inference in high-dimensional settings has emerged as a critical area of research due to the rapid growth of large-scale and complex datasets in fields such as healthcare, economics, genomics, social sciences, and artificial intelligence. Traditional causal inference methods often struggle when the number of covariates exceeds the sample size or when relationships among variables are nonlinear and highly interactive. This study reviews recent advances in causal inference techniques designed for high-dimensional environments, with particular emphasis on the integration of machine learning approaches. The paper examines foundational concepts of causal identification, treatment effect estimation, confounding adjustment, and variable selection under high-dimensional conditions. It further explores how machine learning algorithms, including regularization methods, random forests, boosting, deep learning, and representation learning, enhance the estimation of causal effects while improving predictive accuracy and scalability. The review also discusses modern frameworks such as double machine learning, causal forests, targeted maximum likelihood estimation, and Bayesian causal models, highlighting their theoretical foundations and practical applications. Key challenges, including interpretability, model misspecification, hidden confounding, data heterogeneity, and computational complexity, are critically analyzed. In addition, the study evaluates emerging applications of high-dimensional causal inference in precision medicine, policy evaluation, recommendation systems, and financial analytics. The findings indicate that integrating machine learning with causal inference significantly improves robustness and efficiency in complex data environments, although concerns regarding transparency and generalizability remain substantial. The study concludes that future research should prioritize explainable causal machine learning models, fairness-aware inference methods, and scalable algorithms capable of handling dynamic and streaming data. This review contributes to the growing body of knowledge by synthesizing theoretical developments, methodological innovations, and emerging research directions in high-dimensional causal inference.