Yuqi Pan
ABSTRACT
We consider a decision aggregation problem with two experts who each make a binary recommendation after observing a private signal about an unknown binary world state. An agent, who does not know the joint information structure between signals and states, sees the experts' recommendations and aims to match the action with the true state. Under the scenario, we study whether supplemented additionally with second-order information (each expert's forecast on the other's recommendation) could enable a better aggregation.
We adopt a minimax regret framework to evaluate the aggregator's performance, by comparing it to an omniscient benchmark that knows the joint information structure. With general information structures, we show that second-order information provides no benefit -- no aggregator can improve over a trivial aggregator, which always follows the first expert's recommendation. However, positive results emerge when we assume experts' signals are conditionally independent given the world state. When the aggregator is deterministic, we present a robust aggregator that leverages second-order information, which can significantly outperform counterparts without it. Second, when two experts are homogeneous, by adding a non-degenerate assumption on the signals, we demonstrate that random aggregators using second-order information can surpass optimal ones without it. In the remaining settings, the second-order information is not beneficial. We also extend the above results to the setting when the aggregator's utility function is more general.
Supplemental Material
- Itai Arieli, Yakov Babichenko, and Rann Smorodinsky. 2018. Robust forecast aggregation. Proceedings of the National Academy of Sciences, Vol. 115, 52 (2018), E12135--E12143.Google Scholar
Cross Ref
- Itai Arieli, Yakov Babichenko, and Rann Smorodinsky. 2020. Identifiable information structures. Games and Economic Behavior , Vol. 120 (2020), 16--27.Google ScholarCross Ref
- Itai Arieli, Yakov Babichenko, Inbal Talgam-Cohen, and Konstantin Zabarnyi. 2023. Universally Robust Information Aggregation for Binary Decisions. arXiv preprint arXiv:2302.03667 (2023).Google Scholar
- Yakov Babichenko and Dan Garber. 2021. Learning optimal forecast aggregation in partial evidence environments. Mathematics of Operations Research , Vol. 46, 2 (2021), 628--641.Google ScholarDigital Library
- Jonathan Baron, Barbara A Mellers, Philip E Tetlock, Eric Stone, and Lyle H Ungar. 2014. Two reasons to make aggregated probability forecasts more extreme. Decision Analysis, Vol. 11, 2 (2014), 133--145.Google ScholarDigital Library
- Gabriel Carroll. 2017. Robustness and separation in multidimensional screening. Econometrica, Vol. 85, 2 (2017), 453--488.Google ScholarCross Ref
- Yi-Chun Chen, Manuel Mueller-Frank, and Mallesh M Pai. 2021. The wisdom of the crowd and higher-order beliefs. arXiv preprint arXiv:2102.02666 (2021).Google Scholar
- Robert T Clemen and Robert L Winkler. 1986. Combining economic forecasts. Journal of Business & Economic Statistics , Vol. 4, 1 (1986), 39--46.Google ScholarCross Ref
- Henrique De Oliveira, Yuhta Ishii, and Xiao Lin. 2021. Robust merging of information. arXiv preprint arXiv:2106.00088 (2021).Google Scholar
- Morris H DeGroot. 1974. Reaching a consensus. Journal of the American Statistical association, Vol. 69, 345 (1974), 118--121.Google ScholarCross Ref
- Yongkang Guo, Jason D Hartline, Zhihuan Huang, Yuqing Kong, Anant Shah, and Fang-Yi Yu. 2024. Algorithmic Robust Forecast Aggregation. arXiv preprint arXiv:2401.17743 (2024).Google Scholar
- Wei He and Jiangtao Li. 2022. Correlation-robust auction design. Journal of Economic Theory , Vol. 200 (2022), 105403.Google ScholarCross Ref
- Hadi Hosseini, Debmalya Mandal, Nisarg Shah, and Kevin Shi. 2021. Surprisingly Popular Voting Recovers Rankings, Surprisingly! arXiv preprint arXiv:2105.09386 (2021).Google Scholar
- Victor Richmond R Jose and Robert L Winkler. 2008. Simple robust averages of forecasts: Some empirical results. International journal of forecasting , Vol. 24, 1 (2008), 163--169.Google ScholarCross Ref
- Yuqing Kong. 2024. Peer Expectation in Robust Forecast Aggregation: The Possibility/Impossibility. arxiv: 2402.06062 [cs.GT]Google Scholar
- Yuqing Kong, Yunqi Li, Yubo Zhang, Zhihuan Huang, and Jinzhao Wu. 2022. Eliciting thinking hierarchy without a prior. Advances in Neural Information Processing Systems , Vol. 35 (2022), 13329--13341.Google Scholar
- Richard P Larrick, Albert E Mannes, and Jack B Soll. 2012. The social psychology of the wisdom of crowds. In Social judgment and decision making. Psychology Press, 227--242.Google Scholar
- Gilat Levy and Ronny Razin. 2022. Combining forecasts in the presence of ambiguity over correlation structures. Journal of Economic Theory , Vol. 199 (2022), 105075.Google ScholarCross Ref
- Marcellin Martinie, Tom Wilkening, and Piers DL Howe. 2020. Using meta-predictions to identify experts in the crowd when past performance is unknown. Plos one, Vol. 15, 4 (2020), e0232058.Google ScholarCross Ref
- Eric Neyman and Tim Roughgarden. 2022. Are you smarter than a random expert? The robust aggregation of substitutable signals. In Proceedings of the 23rd ACM Conference on Economics and Computation. 990--1012.Google ScholarDigital Library
- Asa B Palley and Ville A Satop"a"a. 2023. Boosting the wisdom of crowds within a single judgment problem: Weighted averaging based on peer predictions. Management Science (2023).Google Scholar
- Asa B Palley and Jack B Soll. 2019. Extracting the wisdom of crowds when information is shared. Management Science, Vol. 65, 5 (2019), 2291--2309.Google ScholarDigital Library
- Yuqi Pan, Zhaohua Chen, and Yuqing Kong. 2023. Robust Decision Aggregation with Second-order Information. arXiv preprint arXiv:2311.14094 (2023).Google Scholar
- Drazen Prelec. 2004. A Bayesian truth serum for subjective data. science, Vol. 306, 5695 (2004), 462--466.Google Scholar
- Dravz en Prelec, H Sebastian Seung, and John McCoy. 2017. A solution to the single-question crowd wisdom problem. Nature, Vol. 541, 7638 (2017), 532--535.Google Scholar
- Roopesh Ranjan and Tilmann Gneiting. 2010. Combining probability forecasts. Journal of the Royal Statistical Society Series B: Statistical Methodology, Vol. 72, 1 (2010), 71--91.Google ScholarCross Ref
- David M Rothschild and Justin Wolfers. 2011. Forecasting elections: Voter intentions versus expectations. Available at SSRN 1884644 (2011).Google Scholar
- Grant Schoenebeck and Biaoshuai Tao. 2021. Wisdom of the crowd voting: Truthful aggregation of voter information and preferences. Advances in Neural Information Processing Systems , Vol. 34 (2021), 1872--1883.Google Scholar
- James H Stock and Mark W Watson. 2004. Combination forecasts of output growth in a seven-country data set. Journal of forecasting , Vol. 23, 6 (2004), 405--430.Google ScholarCross Ref
- Juntao Wang, Yang Liu, and Yiling Chen. 2021. Forecast aggregation via peer prediction. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9. 131--142.Google ScholarCross Ref
- Tom Wilkening, Marcellin Martinie, and Piers DL Howe. 2022. Hidden experts in the crowd: Using meta-predictions to leverage expertise in single-question prediction problems. Management Science, Vol. 68, 1 (2022), 487--508.Google ScholarDigital Library
- Andrew Chi-Chih Yao. 1977. Probabilistic computations: Toward a unified measure of complexity. In 18th Annual Symposium on Foundations of Computer Science (sfcs 1977). IEEE Computer Society, 222--227. ioGoogle Scholar
Index Terms
- Robust Decision Aggregation with Second-order Information
Recommendations
Second-Order Democratic Aggregation
Computer Vision – ECCV 2018AbstractAggregated second-order features extracted from deep convolutional networks have been shown to be effective for texture generation, fine-grained recognition, material classification, and scene understanding. In this paper, we study a class of ...
Decision aggregation with reliability propagation
AbstractPeople often make decisions differently, even when faced with the same decision-making scenario and objectives, due to their varying abilities to access, process, and comprehend information relevant to the decisions at hand. To reconcile these ...
Highlights- We quantify information reliability to unify crowd-sourced intelligence.
- We model reliability propagation by simultaneously learning relevance graphs.
- Our approach can provide insights into crowd-sourced information reliability.
Universally Robust Information Aggregation for Binary Decisions
EC '23: Proceedings of the 24th ACM Conference on Economics and ComputationWe study a setting with a decision maker making a binary decision by aggregating information from symmetric agents. Each agent provides the decision maker a recommendation depending on her private signal about the hidden state. We assume that agents ...
Comments