Non-asymptotic penalization criteria formixture-of-experts regression models withGaussian gating functions

Mixture-of-experts (MoE) models are a popular framework formodeling heterogeneity in data, for both regression and classificationproblems in statistics and machine learning, due to their flexibility andthe abundance of statistical estimation and model choice tools. Suchflexibility comes from allowing the mixture weights (i.e, the gatingfunctions) in the MoE model to depend on the explanatory variables,along with the experts (i.e, the component densities). This permitsthe modeling of data arising from more complex data generating pro-cesses, compared to the classical finite mixtures and finite mixtures ofregression models, whose mixing parameters are independent of thecovariates. The use of MoE models in a high-dimensional setting,when the number of explanatory variables can be much larger thanthe sample size (i.e.,pn), is challenging from the computationaland theoretical points of view, where the literature still lacks resultsfor dealing with the curse of dimensionality, in both the statisticalestimation and feature selection problems. We aim at estimating thenumber of components of this mixture, as well as the complexity ofthe regression relationship using a penalized maximum likelihood ap-proach. To this end, we provide both a weak oracle inequality and anl1-oracle inequality for MoE regression models with Gaussian gatingfunctions.