Sparklines for questions – Replication Markets

Sparklines for trades

Below for your interest are sparkline plots of the Citation and the No Publication preprint markets, to give a sense of trading activity for claims that had at least 5 trades. Notes:

The X axes are simply trade sequences – they emphasize changes by hiding gaps and give no sense of when the trades happened. Claims with more trades have longer sequences.
The Y axes are scaled to that claim’s high/low – again to emphasize changes. In some cases these only differ by a few %, so the “change” doesn’t mean much. Always check the Hi/Lo values.
These haven’t been copiloted – but we’ll make code and data available shortly so you can make your own.

As expected, many of the NoPub claims get reduced to small values and stay there — after all we knew a hundred papers had been published. But see the note below for some very surprising last-minute behavior by one forecaster.

"Citations" Claims

The goal was to estimate the relative citation rank of each claim, combining citations to the preprint and the publication, as determined by Google Scholar one year from original upload. This is in general a completely intractable combinatorial problem, so we treated each preprint independently. Ideally closing prices are pretty evenly distributed from 0..100. Hard to evaluate that here, but you can see how papers “negotiated” their position.

"No Publication" Markets

Note the large number of claims that were reduced near 0% because they are already published, but then have last-minute “hockey stick” jumps back to about 50%. Strange. These appear to have been done by cthietje83, with typical single trades 1% ➛ 45%. But why buy so many surely losing shares? Yes they’re cheap, but for good reason! Theories:

Confusion: trades like this will temporarily inflate your leaderboard score, but that’s illusory – it’s unrelated to actual payout.
Long Shot: Should one of these get retracted (or have been a mistake), that’s a lot of shares bought for cheap. OK, but the expected value is terrible.
Insider Knowledge: Maybe they know something about these papers. But that would be more believable if it were one or two.
Mischief/Sabotage/Randomness: It certainly messes up our “market closing price” forecast.
Cheating: Dumping points to provide them to someone else. We’ll have to check that.

Had we a realtime sparkline dashboard, or similar, it might have been easier for other traders to cooperate to correct in time.

Matthew Rodriguez and Charles Twardy

Twitter feed is not available at the moment.

There is NO COST to participate in Replication Markets.

This research is supported (in part) by the Fetzer Franklin Fund of the John E. Fetzer Memorial Trust. It uses a platform developed for DARPA SCORE, and some staff are supported by SCORE while working on this. We are grateful for their support.