Big Data-Driven Health Risk Stratification: A Health Index-Based Approach Using Feature Importance and PySpark

Oluwasegun Abiodun Abioye, Martins Ekata Irhebhude — Mon, 24 Mar 2025 00:00:00 +0000

Health risk stratification is crucial for preventive healthcare, yet existing models often rely on binary classification generalized disease prediction, neglecting personalized health indicators and graded risk levels. Many studies apply feature selection techniques like Relief and Univariate Selection without quantifying the weighted impact of features. To address these gaps, this study introduces a Big Data-driven Health Index (HI) framework using PySpark for scalable health risk stratification. The HI is computed as a weighted sum of health-related features using SHAP Analysis, XGBoost, Random Forest, and Correlation Analysis. PySpark enables efficient processing of large-scale health data, and individuals are classified into Low and High Risk. Optimal classification thresholds are determined using the Youden Index from the ROC curve to balance sensitivity and specificity. Personalized health recommendations are generated based on risk categories to guide preventive interventions. Performance evaluation reveals that Correlation Analysis achieves 100% precision and 98.90% recall, outperforming other methods. SHAP prioritizes recall but has low precision, while XGBoost and Random Forest improve precision but struggle with recall. By leveraging Big Data techniques with PySpark, this study enhances computational efficiency, scalability, and classification accuracy, addressing prior research limitations and providing a robust data-driven approach to personalized health monitoring.

Aspect-Based Sentiment Analysis on E-commerce Reviews using BiGRU and Bi-Directional Attention Flow

Tue, 01 Apr 2025 00:00:00 +0000

Aspect-based sentiment Analysis (ABSA) is vital in capturing customer opinions on specific e-commerce products and service attributes. This study proposes a hybrid deep learning model integrating Bi-Directional Gated Recurrent Units (BiGRU) and Bi-Directional Attention Flow (BiDAF) to perform aspect-level sentiment classification. BiGRU captures sequential dependencies, while BiDAF enhances attention by focusing on sentiment-relevant segments. The model is trained on an Amazon review dataset with preprocessing steps, including emoji handling, slang normalization, and lemmatization. It achieves a peak training accuracy of 99.78% at epoch 138 with early stopping. The model delivers a strong performance on the Amazon test set across four key aspects: price, quality, service, and delivery, with F1 scores ranging from 0.90 to 0.92. The model was also evaluated on the SemEval 2014 ABSA dataset to assess generalizability. Results on the restaurant domain achieved an F1-score of 88.78% and 83.66% on the laptop domain, outperforming several state-of-the-art baselines. These findings confirm the effectiveness of the BiGRU-BiDAF architecture in modeling aspect-specific sentiment across diverse domains.

Journal of Computing Theories and Applications

Big Data-Driven Health Risk Stratification: A Health Index-Based Approach Using Feature Importance and PySpark

Aspect-Based Sentiment Analysis on E-commerce Reviews using BiGRU and Bi-Directional Attention Flow