WP-277: Fiona Burlig, Louis Preonas, and Matt Woerman, "Panel Data and Experimental Design" (January 2017) | Full Paper
How should researchers design experiments to detect treatment effects with panel data? In this paper, we derive analytical expressions for the variance of panel estimators under non-i.i.d error structures, which inform power calculations in panel data settings. Using Monte Carlo simulation, we demonstrate that, with corrected errors, traditional methods for experimental design results in experiments that are incorrectly powered with proper inference. Failing to account for serial correlation yields overpowered experiments in short panels and underpowered experiments in long panels. Using both data from a randomized experiment in China and a high-frequency datatset of U.S. electricity consumption, we show that these results hold in real-world settings. Our theoretical results enable us to achieve correctly powered experiments in both simulated and real data. This paper provides researchers with the tools to design well-powered experiments in panel data settings.