Solar Energy Prediction: An International Contest to Initiate Interdisciplinary Research on Compelling Meteorological Problems

As meteorological observing systems and models grow in complexity and number, the size of the data becomes overwhelming for humans to analyze using traditional techniques. Computer scientists, and specifically machine learning and data mining researchers, are developing frameworks for analyzing big data. The AMS Committee on Artificial Intelligence and its Applications to Environmental Science aims to bring AI researchers and environmental scientists together to increase the synergy between the two. The AI committee has sponsored 4 previous contests on a variety of meteorological problems including wind energy, storm classification, winter hydrometeor classification, and air pollution, with the goal of bringing together the two fields of research. Although these were successful, the audience was limited to existing environmental science researchers (usually 10-20 teams of people primarily within the AMS community). For the 2013/14 contest, we expanded to a global audience by focusing on the compelling problem of solar energy prediction and by having the established forum Kaggle host our contest. Using this forum, we had over 160 teams from all around the world participate. Improved solar energy forecasting is a necessary component of making solar energy a viable alternative power source. This paper summarizes our experiences in the 2013/14 contest, discusses the data in detail, and presents the winning prediction methods. The contest data come from the NOAA/ESRL Global Ensemble Forecasting System Reforecast Version 2 and the Oklahoma Mesonet with sponsorship from EarthRisk Technologies. All winning methods utilized gradient boosted regression trees but differed in parameter choices and interpolation methods.