Follow us
Publications Commentary Research People Events News Resources and Videos About IFS
Home Publications A lava attack on the recovery of sums of dense and sparse signals

A lava attack on the recovery of sums of dense and sparse signals

Cemmap Working Paper CWP56/15

Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of non-zero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small non-zero parameters. We consider a generalization of these two basic models, termed here a “sparse + dense” model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.

More on this topic

IFS Working Paper W22/22
This paper describes methods to optimally choose the number of treatment and control clusters, and the number of units within treatment and control clusters, allowing for full flexibility.
Cemmap Working Paper CWP09/22
We compare two approaches to using information about the signs of structural shocks at specific dates within a structural vector autoregression (SVAR): imposing ‘narrative restrictions’ (NR) on the shock signs in an otherwise set-identified SVAR; and casting the information about the shock ...
IFS Working Paper W22/13
We specify and estimate a lifecycle model of consumption, housing demand and labor supply in an environment where individuals may file for bankruptcy or default on their mortgage.
Cemmap Working Paper CWP07/22
Economists are obsessed with rankings of institutions, journals, or scholars according to the value of some feature of interest.
Cemmap Working Paper CWP04/22
It is often desired to rank different populations according to the value of some feature of each population. For example, it may be desired to rank neighbourhoods according to some measure of intergenerational mobility or countries according to some measure of academic achievement.