An alternative method to developing a stock market index: Machine learning implementation using higher moments and asset liquidity

Abstract

Stock market indexes have a very significant role in asset pricing models, particularly in the Capital Asset Pricing Model (CAPM), as a proxy for the entire wealth in the economy. There is widespread agreement in academia and industry that popular indices are inadequate at reflecting the statistical properties of the market portfolio. This study differs from traditional approaches in two aspects: the index component selection and the components’ weight assignment. K-Means machine learning technique is applied through stock moments to exclude outliers' impact on the index and liquidity for index component selection. Principal Component Analysis (PCA) was used to determine index component weights in addition to equal weighting and market-cap weighting to reduce value and growth stocks disproportionate influence on the index. Except for skewness, the PCA-based weighting index results are remarkably similar to the Center for Research in Security Prices (CRSP) market-cap weight index. The PCA-based weighted index has a significantly greater negative skew than other prominent indices.