Replication Markets: COVID-19 Rules

Official Contest Rules v. 1.3.

The Replication Markets: COVID-19 Prize Contest (the “Contest”) is sponsored by the Replication Markets Project (the “Sponsor”), having an address of:


    2551 Dulles View Drive

    Suite 700

    Herndon, Virginia 20171

Change Log:

  • 2020-11-10: all markets will pay at the end (Sep. 2021). It was not feasible to resolve Publication questions monthly. (Surveys still pay Dec. 2020.)
  • 2020-11-12: extend finish by one day to Nov. 19, because of downtime.
  • 2020-11-15: extend finish to Nov. 20 because of weekend downtime and lack of liquidity; detail how Citation Rank claims resolve.

Summary for humans:

  • No purchase necessary to win.
  • You must be 18 years old and not affiliated with the Sponsor.
  • You may participate even if declining or ineligible for prizes.
  • One user, one account, except by written agreement.
  • Follow the rules. Play nice. Don’t cheat. Don’t try to lose.
  • Contest runs October 28, 2020 through November 18 20, 2020.
  • Contest will forecast (1) publication, (2) citation, (3) replication, and (4) usefulness for the Top-400 claims from COVID-19 research, using both surveys and markets.
    • Surveys will ask all three questions, and pay out by the end of 2020.
    • Markets will only forecast publication and citation.
    • Markets pay out 1 year from preprint upload.
  • A total of $14,520 in prizes will be awarded. Prizes are divided as follows.
    • Surveys $3,600: $90 per survey batch, paid as $30, $25, $20, and $15 to the 1st, 2nd, 3rd, and 4th place for that batch. Each batch has 10 questions. Participants may win more than one batch.
    • Markets $10,920: paid proportional to winning shares.
  • The Sponsor will use PayPal to distribute funds to all winners.
  • Recipients may be required to verify identity.
  • Taxes and fees are paid by recipients.
  • Usual liability limitations, and final authority with the Sponsor.



The following individuals are also ineligible to participate in the Contest: employees of the Sponsor and any officers, directors, investors, agents, and representatives of the Sponsor and their parent organizations, subsidiaries, or affiliates, including any sponsors and affiliates and advertising and promotion agencies of the Sponsor, and employees of the Fetzer Fund (“Ineligible Affiliates”). This restriction also applies to any immediate family members (defined to include parents, spouse, or children) of those Ineligible Affiliates. The contest is also not available to any Users previously removed from the Contest by the Sponsor. The Sponsor reserves the right at its sole and absolute discretion to limit the number of contests you may participate in at any given time, should the Sponsor offer other contests.

Contest Period:

The contest will occur from October 28, 2020 to November 18 20, 2020, as announced.

How to enter:

Prior to or during the Contest Period, visit and either register or log into your account. Completely and accurately submit the required online member registration form, including a reliable email address. Take the required intake survey and optionally the other intake surveys. Then make forecasts to receive virtual points as explained below and in our pre-registration.

You may only use a single account in the Contest during the duration of the Contest Period. If you have more than one account, notify the Sponsor by email ( to have the other(s) deactivated or monitored.

Please note that by registering an Account, you will be subject to all notices posted on the Site, including but not limited to the Site’s Informed Consent and Privacy Policy.

By entering the Contest, you agree that your Portfolio and any other information provided by you or collected by the Sponsor in connection with the Contest may be used by the Sponsor in accordance with the Sponsor’s Privacy Policy.

By registering for the Contest, all entrants agree to abide by these Official Contest Rules.

By registering for the Contest, you agree not to allow any other person to use or access your account for any purpose. You are responsible for maintaining the security of your account information, including your password.

How Not to Play:

The Sponsor reserves the right to penalize, suspend, disable, and/or destroy the account(s) of any entrant that it believes in good faith has engaged in any negative or cheating behavior, attempted to interfere with or undermine the Contest or the accuracy of the forecasts, or damage the website, even if those attempts are unsuccessful.

These attempts may be a violation of criminal or civil laws. Should such an attempt be made, the Sponsor reserves the right to seek damages from any such individual to the fullest extent permitted by applicable law and/or to take other actions deemed necessary by the Sponsor in its sole discretion to protect its rights.

Examples of violations include:

  • Registering more than once, creating and/or managing more than one account without written authorization from the Sponsor for legitimate purposes.
  • Coordinating one’s survey or trading activity with any other entrant(s), other than to maximize accuracy-based gains.
  • Providing incomplete, non-current, false, or inaccurate data in the registration form.
  • Engaging, or planning or threatening to engage in any activity which may result in restricting or inhibiting any other entrant from using and enjoying the Contest, including, but not restricted to boycotts, law suits, computer attacks, and harassment.
  • Using third party software robots, scripts, or any other automated system to participate in the Contest outside the approved scripting API and terms, or in a manner that interferes with other forecasters or the accessibility of the system.
  • Engaging in any kind of transaction meant primarily to transfer virtual funds (points) from one account to another.
  • Engaging in any kind of purposefully losing behavior with respect to true outcomes. This includes most forms of collusion and manipulation in surveys and markets.
  • Using the discussion forums or comment areas to spread unlawful, threatening, abusive, libelous, defamatory, obscene, vulgar, pornographic, profane or indecent information of any kind.
  • Using the discussion forums or comment areas for off-topic criticism of the Sponsor, its partners, staff, or other affiliated parties. (As opposed for example to on-topic objections to a judgment call, feature, or rule.)
  • Using the discussion forums to transmit any kind of advertising.

The nature and timing of penalties and all other determinations will be entirely at the Sponsor’s sole discretion and may be applied without prior warning.


A minimum of $14,520 in prizes will be awarded, and announced as they are determined: survey prizes by the end of 2020, and markets after the true outcomes are revealed. Winners will receive money via PayPal. Requirements for each prize and amount of prize are described above (see Eligibility). Any  pertinent revisions and clarifications will be recorded in a ChangeLog linked from these Rules.

Survey prize amounts are per batch, as described above. Market prizes pay out after all resolve (Sep. 2021), proportional to winning shares. 

Decisions of the Sponsor are final and binding with respect to all matters related to the Contest. In no event shall the Sponsor be obligated to award any prizes other than the Prizes specified in these Official Contest Rules. The Prizes are non-transferable and no substitutions are permitted. Winners shall be responsible and liable for all federal, state and local taxes on the value of the Prizes. Transfer fees (if any) will be deducted from the prize amount. The Sponsor may consider requests to defer small payments.

Winners lists will be posted to the Replication Markets blog, by username. To receive a complete list of winners or a copy of the Official Contest Rules, within six(6) months after the end of the Contest Period, send a self-addressed, stamped envelope to the Sponsor address listed above. Specify winner’s list or rules on your request.

Prize Selection:

Survey Prizes (Surrogate Scoring)

The prize amounts for surrogate scoring are described above. Prizes are awarded based on a pre-registered peer prediction (surrogate score) method, calculated on each survey question and aggregated over batches of 10 surveys. Payout will happen after markets close, and by the end of 2020.With high probability, top ranks will go to the most accurate forecasters.

Market Prizes:

Market prize amounts are described above. Prizes are awarded proportional to winning shares in the publication and citation markets. All markets pay out in Sep. 2021 after the last claim resolves.

Every action in the market buys or sells shares in some outcome like “Yes” or “No”. Shares are shown. Suppose you have exactly 30 winning shares: 10 correct Yes in one claim, and 20 correct No in another. If there are total 1,000 winning shares that month, you will earn 3% of that month’s prize pool.

(Citation markets resolve as mixtures: a paper in the 75th %ile will count each Yes share as 0.75 and each No share as 0.25.)

While the market is open, you can sell shares to free up points to spend elsewhere. If the market has moved in your favor, your sold shares will earn a profit. If it moved against you, they will sell at a loss. If it is where you left it, selling your shares returns you to where you started.

We only care about trades that change the price, so you can’t hold opposing shares. It will sell your Yes  shares before buying No shares, and conversely.  


Winners will be selected after all eligible activity, all ranks or points are calculated, all corrections (if any) are made, and all site maintenance is performed. The Sponsor will confirm eligibility and compliance with these Official Contest Rules. Winners must be able to receive payments, and comply with any required tax or legal notifications.

The Contest will be conducted under the supervision of the Sponsor. The decisions of the Sponsor are final and binding in all matters relating to this Contest. Winners will be notified by email; however, the Sponsor reserves the right to determine an alternate method of notification. Winners must claim their Prize within 30 days after the date of notification of such Prize. A Winner’s failure to respond to the Prize notification within the specified 30 days will be considered such Winner’s forfeiture of the Prize and an alternate Winner may be selected from the pool of eligible entries. If an entrant is found to be ineligible, an alternate Winner may also be selected from the pool of eligible entries.

In the event of a dispute regarding the identity of the potential Winner, the potential Winner will be deemed to be the person in whose name the applicable Account is registered, and proof of being the authorized account holder may be required. Decision of the Sponsor as to the identity of the entrant shall be final.

Survey winners are selected after the markets are closed, as described above. They resolve ahead of claim resolution, but with high probability, the top ranks will also be the most accurate.

Claim Resolution (Market Prizes):

Publication contracts resolve within 1 year of upload to the preprint server as one of: (1) Not Published; (2) Yes, in a Journal with Impact Factor below 10, and (3) Yes, in a Journal with Impact factor at least 10.  Sponsor will use bioRxiv, Web of Science and related sources to determine publication and Impact Factor.

Sponsor will resolve Citation contracts with Google scholar citation counts 1 year from the month of preprint upload.  

In the case of any disagreement, ambiguity or differing interpretations among sources, or of disputed outcomes or any other valuation-related issues, the Sponsor reserves the right to make final and binding decisions on all matters, and such decisions will not be subject to appeal. The Sponsor provides a 48-hour discussion period after nominating a potential resolution. Forecasters are encouraged to register any objections and supporting arguments during that period.

The Sponsor may extend the advertised expiration of a forecast question if the outcome is still unknown / unknowable as of the expiration date. 


Market prizes are paid proportional to winning shares in resolving claims. Shares are earned by trading in the market. The market uses a logarithmic markets scoring rule (LMSR) engine with base 2 and liquidity parameter of b = 100. For a trade on the correct outcome:

     gain = 100 x log2 (p’ / p)

where p is the old probability and p’ is the new probability, for that outcome. If p’ > p, gain is positive. If p’ < p, gain is negative.  

Unlike other markets, all bets are hedged: if you increase Yes, you also reduce No. The cost is the minimum gain (i.e. the maximum loss), and the shares in an outcome are g – cost, where g is the gain should that outcome happen. Shares “pay 1 point if true”; in the worst case, you have 0 shares in an outcome.  

Markets can resolve at intermediate values. Nearly all Citation Rank markets do: a paper in the 75th %ile values Yes shares at 0.75 and No shares at 0.25.

The default leaderboard calculates the expected value of your portfolio, with the expectation taken at current market prices. Therefore, unless you have no investments, it fluctuates with the market. Your available points do not fluctuate. But you must invest them by market close or lose them. 

There is NO COST to participate in Replication Markets.

This research is supported (in part) by the Fetzer Franklin Fund of the John E. Fetzer Memorial Trust. It uses a platform developed for DARPA SCORE, and some staff are supported by SCORE while working on this. We are grateful for their support.