Platform update: Leaderboards

We have added leaderboards to track Markets vs. Surveys and per-Round scores. 

screen shot of leaderboard dropdown menu

This should give all forecasters (not just those who joined at the beginning of the project) a better idea of where they stand.

The “Overall Market” is the default. It shows the total Expected Value of your entire market portfolio, using the current market prices as the best guide.

Though we have to qualify “current”: this value is updated every four hours. But we are not chasing the needles, right?

Here is the “Overall Market” leaderboard (as of this writing). This includes a forecaster’s current unspent points, so it is optimistic.

Notice the rm_test account has 300 unspent points which will vanish unused at the end of the round. Use ’em or lose ’em: points reset at the beginning of each round. 

overall market leaderboard at time of blog post

Here is the leaderboard for the Round 1 Market, which closed on September 29. Share prices and rankings are fixed at a round’s closing. 

As of this writing, Round 1 and Round 2 leaderboards are fixed. Anyone who recently joined the project has the chance to see their name at the top of Round 3’s.

R1 leaderboard

After each round, we convert the survey peer prediction score into batch ranks, then create a score based on overall batch ranks within the round.

R1 Survey leaderboard
R2 survey leaderboard

Here you can see that the Round 1 Surveys were relatively uncontested compared to Round 2. 

Round 3 Surveys opened on Monday. The more batches a forecaster completes, the more likely they will appear on the leaderboard.

We need forecasters to complete as many surveys as possible. It is the combined individual assessments of our forecasters that sets the baseline for the crowdsourced opinion in the Markets. 

Completed surveys are good for our research and good for our forecasters: we get data, and forecasters get a chance to win money for their accuracy. 

Round 3 Surveys are open until November 10 at 23:59 UTC.

Round 3 Markets will be active November 11, 12:00 UTC through November 24, 23:59 UTC.

Want to see where you stand in the leaderboards?

Contribute to the discussion...

This site uses Akismet to reduce spam. Learn how your comment data is processed.

May we send you invitations to future research projects? 

Add your email address to our low-volume distribution list.

Note: We cannot re-use participant lists for new recruiting. Please opt-in (above) for occasional announcements about related future studies.

We’re sorry to see you go! Please visit our social media sites.

This site uses cookies to provide you with a better browsing experience.

Visit our Privacy Policy for more information.