ABSTRACT
Large-scale online platforms launch hundreds of randomized experiments (a.k.a. A/B tests) every day to iterate their operations and marketing strategies, while the combinations of these treatments are typically not exhaustively tested. It triggers an important question of both academic and practical interests: Without observing the outcomes of all treatment combinations, how to estimate the causal effect of any treatment combination and identify the optimal treatment combination? We develop a novel framework combining deep learning and double machine learning to estimate the causal effect of any treatment combination for each user on the platform when observing only a small subset of treatment combinations. Our proposed framework (called debiased deep learning, DeDL) exploits Neyman orthogonality and combines interpretable and flexible structural layers in deep learning. We prove theoretically that this framework yields consistent and asymptotically normal estimators under mild assumptions, thus allowing for identifying the best treatment combination when only observing a few combinations. To empirically validate our method, we then collaborate with a large-scale video-sharing platform and implement our framework for three experiments involving three treatments where each combination of treatments is tested. When only observing a subset of treatment combinations, our DeDL approach significantly outperforms other benchmarks to accurately estimate and infer the average treatment effect (ATE) of any treatment combination and to identify the optimal treatment combination.
Index Terms
- Deep Learning Based Causal Inference for Large-Scale Combinatorial Experiments: Theory and Empirical Evidence
Recommendations
An Overview of Deep Reinforcement Learning
CACRE2019: Proceedings of the 2019 4th International Conference on Automation, Control and Robotics EngineeringAs a new machine learning method, deep reinforcement learning has made important progress in various fields of people's production and life since it was proposed. However, there are still many difficulties in function design and other aspects. Therefore,...
Deep reinforcement learning boosted by external knowledge
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied ComputingRecent improvements in deep reinforcement learning have allowed to solve problems in many 2D domains such as Atari games. However, in complex 3D environments, numerous learning episodes are required which may be too time consuming or even impossible ...
Comments