June 2023 Optimal high-dimensional and nonparametric distributed testing under communication constraints
Botond Szabó, Lasse Vuursteen, Harry van Zanten
Author Affiliations +
Ann. Statist. 51(3): 909-934 (June 2023). DOI: 10.1214/23-AOS2269

Abstract

We derive minimax testing errors in a distributed framework where the data is split over multiple machines and their communication to a central machine is limited to b bits. We investigate both the d- and infinite-dimensional signal detection problem under Gaussian white noise. We also derive distributed testing algorithms reaching the theoretical lower bounds.

Our results show that distributed testing is subject to fundamentally different phenomena that are not observed in distributed estimation. Among our findings we show that testing protocols that have access to shared randomness can perform strictly better in some regimes than those that do not. We also observe that consistent nonparametric distributed testing is always possible, even with as little as one bit of communication, and the corresponding test outperforms the best local test using only the information available at a single local machine. Furthermore, we also derive adaptive nonparametric distributed testing strategies and the corresponding theoretical lower bounds.

Funding Statement

Cofunded by the European Union (ERC, BigBayesUQ, project number: 101041064). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them.

Acknowledgments

We would like to thank Elliot H. Lieb for a helpful comment regarding the proof of Lemma 10.1.

Citation

Download Citation

Botond Szabó. Lasse Vuursteen. Harry van Zanten. "Optimal high-dimensional and nonparametric distributed testing under communication constraints." Ann. Statist. 51 (3) 909 - 934, June 2023. https://doi.org/10.1214/23-AOS2269

Information

Received: 1 August 2022; Revised: 1 December 2022; Published: June 2023
First available in Project Euclid: 20 August 2023

MathSciNet: MR4630935
zbMATH: 07732734
Digital Object Identifier: 10.1214/23-AOS2269

Subjects:
Primary: 62F30 , 62G10
Secondary: 62F03

Keywords: Distributed methods , Hypothesis testing , minimax optimal , nonparametric

Rights: Copyright © 2023 Institute of Mathematical Statistics

JOURNAL ARTICLE
26 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

Vol.51 • No. 3 • June 2023
Back to Top