This study proposes an information-theoretic framework for analyzing hybrid machine learning models applied to financial time series within complex systems. The approach integrates Autoregressive Integrated Moving Average (ARIMA), Holt--Winters, and Long Short-Term Memory (LSTM) models, whose forecasts are combined using Extreme Gradient Boosting (XGBoost) as a meta-learner. This hybrid architecture captures linear, seasonal, and nonlinear dynamics inherent in financial data.

A key contribution lies in the use of Kullback--Leibler divergence as an information-theoretic measure to quantify the discrepancy between predicted and observed distributions. Unlike conventional error-based metrics, this approach provides deeper insight into uncertainty and distributional characteristics of model predictions.

The framework is applied to long-term financial time series, including major stock indices such as NIFTY 50, Dow Jones Industrial Average, and S\&P BSE SENSEX. Empirical results demonstrate that the hybrid model achieves improved predictive performance and reduced distributional divergence compared to individual models.

The proposed methodology highlights the potential of combining ensemble learning with information-theoretic measures for uncertainty-aware forecasting and offers a generalizable approach for scientific machine learning in complex dynamical systems.