这篇论文提出了SPDNN方法,专门解决训练数据和目标数据分布不同的问题。它能量化回归和Huber回归,还能处理时间序列依赖数据。理论证明能达到最优收敛速度,挺扎实的。
该论文提出稀疏惩罚深度神经网络(SPDNN)估计器,处理协变量偏移下的非参数分位数和Huber回归。当密度比未知时,采用两步预训练:先构建最小二乘SPDNN估计密度比,再用于重加权回归。在Hölder光滑函数类中,SPDNN估计器能达到(对数因子内)极小化最优收敛率,适用于i.i.d.和多种时间序列模型(如φ-mixing、强混合等)。
Adaptive deep nonparametric regression from dependent data under covariate shift
Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, $φ$-mixing, strong mixing, and $\mathcal{C}$-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of Hölder smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.