论文标题
大规模双重惩罚的ANOVA建模
Block-wise Primal-dual Algorithms for Large-scale Doubly Penalized ANOVA Modeling
论文作者
论文摘要
对于多元非参数回归，最近已提出了双重惩罚的ANOVA建模（DPAM），使用分层总变化（HTV）和经验规范，作为对组件函数的惩罚，例如主要效果，例如在函数方向反互作用函数中的主要效果和多向相互作用的函数互动。这两个惩罚扮演着互补的角色：HTV惩罚促进了每个组件函数内基础函数的选择稀疏性，而经验惩罚则促进了组件函数选择时的稀疏性。我们对训练DPAM采用反贴合或最小化，并开发两个合适的原始偶偶联算法，包括批处理和随机版本，以单块优化更新每个组件功能。在我们的环境中，HTV和经验 - 惩罚都在我们的环境中棘手。通过广泛的数值实验，我们在大规模的情况下，与它们的批处理版本和先前的活动算法相比，我们的随机原始二次算法的有效性和优势。
For multivariate nonparametric regression, doubly penalized ANOVA modeling (DPAM) has recently been proposed, using hierarchical total variations (HTVs) and empirical norms as penalties on the component functions such as main effects and multi-way interactions in a functional ANOVA decomposition of the underlying regression function. The two penalties play complementary roles: the HTV penalty promotes sparsity in the selection of basis functions within each component function, whereas the empirical-norm penalty promotes sparsity in the selection of component functions. We adopt backfitting or block minimization for training DPAM, and develop two suitable primal-dual algorithms, including both batch and stochastic versions, for updating each component function in single-block optimization. Existing applications of primal-dual algorithms are intractable in our setting with both HTV and empirical-norm penalties. Through extensive numerical experiments, we demonstrate the validity and advantage of our stochastic primal-dual algorithms, compared with their batch versions and a previous active-set algorithm, in large-scale scenarios.
