We propose a new machine-learning-based approach for forecasting Value-at-Risk (VaR) named CoFiE-NN where a neural network (NN) is combined with Cornish-Fisher expansions (CoFiE). CoFiE-NN can capture non-linear dynamics of high-order statistical moments thanks to the flexibility of a NN while maintaining interpretability of the outputs by using CoFiE which is a well-known statistical formula. First, we explain CoFiE-NN. Second, we compare the forecasting performance of CoFiE-NN with three conventional models using both Monte Carlo simulation and real data. To do so, we employ Long Short-Term Memory (LSTM) as our main specification of the NN. We then apply the CoFiE-NN for different asset classes, with a focus on foreign exchange markets. We report that CoFiE-NN outperfoms the conventional EGARCH-t model and the Extreme Value Theory model in several statistical criteria for both the simulated data and the real data. Finally, we introduce a new empirical proxy for tail risk named tail risk ratio under CoFiE-NN. We discover that the only 20 percent of tail risk dynamics across 22 currencies is explained by one common factor. This is contrasting to the fact that 60 percent of volatility dynamics across the same currencies is explained by one common factor.