Oct 21 2020
10

Surrogate Scoring Rules

By Juntao Wang

Surrogate scoring rules (SSR) is the method we used to score forecasters for accuracy in our survey. We encourage all forecasters to watch our short video about the method. This document has three sections: (1) Overview, (2) Implementation, (3) Example. SSR Overview Surrogate scoring rules enable us to score forecasts’ accuracy using strictly proper scoring…

Jan 02 2020
4

Robustness of Monthly Prizes

By Juntao Wang

Evidence from the first four rounds tl;dr: In our monthly surveys, high participation does not necessarily lead to high prizes – some highly active forecasters earned $0 – suggesting our surrogate score is robust to participation bias. Background The monthly survey prizes are risky.  Unlike markets that pay for ground truth (eventually, and for only…