Open Access
2023 Estimating the proportion of signal variables under arbitrary covariance dependence
X. Jessie Jeng
Author Affiliations +
Electron. J. Statist. 17(1): 950-979 (2023). DOI: 10.1214/23-EJS2119

Abstract

Estimating the proportion of signals hidden in a large number of noise variables is a pervasive objective in scientific research. In this paper, we consider realistic, yet theoretically challenging scenarios with arbitrary covariance dependencies between variables. We quantify the overall level of covariance dependence using mean absolute correlation (MAC), and investigate the performance of a family of estimators across the full range of MAC values. We explore the joint effect of MAC dependence, signal sparsity, and signal intensity on estimator performance, and find that no single estimator in the family performs optimally across all MAC dependence levels. Based on this theoretical insight, we propose a new estimator that is better suited to arbitrary covariance dependencies. Our method compares favorably to several existing methods in a variety of finite-sample settings, including those with strong or weak covariance dependencies and real dependence structures from genetic association studies.

Citation

Download Citation

X. Jessie Jeng. "Estimating the proportion of signal variables under arbitrary covariance dependence." Electron. J. Statist. 17 (1) 950 - 979, 2023. https://doi.org/10.1214/23-EJS2119

Information

Received: 1 November 2021; Published: 2023
First available in Project Euclid: 29 March 2023

MathSciNet: MR4568053
zbMATH: 07690316
Digital Object Identifier: 10.1214/23-EJS2119

Subjects:
Primary: 62F35
Secondary: 62F10

Keywords: Dependence adaptivity , high-dimension data , lower bound estimator , sparse signal

Vol.17 • No. 1 • 2023
Back to Top