Lenka Fiala is a busy woman: she is a full time Ph.D student at Tilburg University, she is a successful forecaster with Replication Markets, and she stays active through circus disciplines. We interviewed Lenka by Skype about her experience in our project.
Tell us a little about yourself and your academic interests and why you got involved with Replication Markets.
I am a naturally curious economist, with a primary focus on experimental economics. I’m also drawn to a variety of related topics, like the economics of education, crime, and health.
I noticed the so-called replication crisis that hit Psychology, and that many people were worried about something similar in Experimental Economics. I find it important to stay informed about what’s going on in my discipline.
Have you been involved in replications or metascience studies?
My academic career started as a research assistant in a meta study. It relates to replicability because you expect to see the same results popping up over and over. It was a great learning experience and solidified the idea that I wanted to go to grad school. I look back at that experience fondly.
I’m involved in a new meta study with some amazing co-authors, and hopefully that can say something about what replicates and what doesn’t.
Let’s talk about your involvement with Replication Markets. You have been a winner in all of our survey rounds, to date. What is your strategy?
I try to do a little bit daily rather than invest a substantial portion of time in one day. For example, when I come home from work and I’m making dinner, it’s something to do while pasta is cooking.
I definitely don’t have time to take all of the surveys, even though I would love to. I normally do at least five batches simply out of curiosity to see what kind of research is out there. If I get a batch that I don’t really like, I’ll stop for the week.
How much time does it take to complete one survey thoughtfully?
It really depends on if I’m familiar with the paper. Often I run across papers that I’ve read before or at least heard of before, and in those cases it goes much quicker. If it’s something I’m not familiar with and I have to open the paper and read their methodology or definitions, it can easily take 20, 30, or 40 minutes.
Often the claims are very short, but if you don’t understand how people define certain concepts, you can’t properly judge the paper.
Do you have any insights into why you are a successful forecaster?
That’s really hard to say. I have no idea how your evaluation function works! I think what I’m doing well is understanding the basics of betting. As long as you’re looking for something that is a function of what other people are doing, even if you believe in the extreme, you can never submit the extreme.
You can expect that some people agree with it and some people disagree with it, so even if your initial belief is that there is no chance that a claim is actually true, you should never submit 0. And likewise for the other end.
It’s a bit like the beauty contest game: even if you are in a room with other economists, and are reasonably sure everybody knows what the equilibrium is, the winning bet will typically not be the prescribed equilibrium strategy.
Do you have any other tips for forecasters?
Don’t just look at the p value, but where the p value is coming from. Look at the sample size, look at the type of test to evaluate the claim, and even if the type of test is appropriate for the type of data that is gathered.
Have you participated in a prediction market before? Do you have a strategy?
This is my first prediction market. It took me weeks to figure out how the market actually works, and it was only in Round 4 that I properly understood what the markets were doing. I finally became much more picky in the market. Only in the next round will I be able to say what kind of strategy I have, if any.
What would you tell other people to encourage them to join the project?
You should join because it is one of the more inspiring ways to procrastinate. On a more serious note, it is an opportunity to put your academic mindset to work for the greater good of science — you can’t say no to that!
Is there anything that you did not expect from participating in Replication Markets?
Honestly, the money is a bonus! I did not expect to be doing so well. I’m investing my winnings into sport equipment that I’ve been wanting to buy for some time, a cyr wheel.
