Appendix
Probabilistic Decision Function
In the PROBABILISTIC case, the probability pA that the principal chooses market A is given as
pA= e10ae10a+ e10b
with a and b being the prices in markets A and B, respectively. The exact function is not revealed to the participants either ex-ante or ex-post, so the exact definition is unlikely to influence the participant too much.
Starting Price Scenarios
Table A1: Scenarios for starting prices of markets
Probability A / Probability B / Price A / Price B67.9 / 32.1 / 81.8 / 37.8
67.9 / 32.1 / 73.1 / 77.7
67.9 / 32.1 / 50 / 26.9
67.9 / 32.1 / 50 / 77.7
67.9 / 32.1 / 22.3 / 50
67.9 / 32.1 / 73.1 / 50
67.9 / 32.1 / 22.3 / 26.9
67.9 / 32.1 / 62.2 / 18.2
Mechanical Turk Users and Low Stakes
To appreciate the value of Mechnical Turk as experimental platform, a couple of points should be noted: (1) Mechnical Turk provides a more representable population than a lab (Paolacci et al. 2010). (2) External validity has been addressed in methodological papers in, e.g., economics (Horton et al., 2011; Amir et al. 2012), political science (Berinsky et al. 2012), and psychology (Buhrmester et al. 2011): These papers report a close match of experimental results from Mechnical Turk and the lab for a wide variety of decision tasks and games. Fehr et al. (2013) present a labor market experiment where stakes vary by factor 10 without substantial behavioral effect. Similarly, Slonim and Roth (1998) use a factor of 25 without substantial behavioral effect for inexperienced subjects. (3) Our payout is above average for Mechnical Turk. In this crowd labor market, it is considered fair and induces conscious behavior. (4) Stakes are about the same for all our treatments and both experiments. While low stakes might be speculated to induce arbitrary behavior, this cannot explain treatment /experiment differences. (5)Low average payouts are standard in real live implementations of decision markets where users are motivated by reputation, which is also a driver in Mechnical Turk.
Amir, O., Rand, D.G., Gal, Y.K., 2012. Economic Games on the Internet: The Effect of $1 Stakes. PLoS ONE 7(2), e31461.
Berinsky, A.J., Huber, G.A., Lenz, G.S., 2012. Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk. Political Analysis 20(3), 351-368.
Buhrmester, M., Kwang, T., Gosling, S.D., 2011. Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality Data? Perspectives on Psychological Science 6(1), 3-5.
Fehr, E., Tougareva, E., Fischbacher, U. 2014. Do high stakes and competition undermine fair behaviour? Evidence from Russia. Journal of Economic Behavior and Organization, 108, 354-363.
Horton, J.J., Rand, D.G., Zeckhauser, R.J., 2011. The online laboratory: conducting experiments in a real labor market. Experimental Economics 14(3), 399-425.
Paolacci, G., J. Chandler and P. Ipeirotis, (2010) “Running Experiments on Amazon Mechanical Turk,” Judgment and Decision Making (5:5), pp. 411–419.
Slonim, R., Roth, A.E., 1998. Learning in High Stakes Ultimatum Games: An Experiment in the Slovak Republic. Econometrica 66(3), 569-596.
Experiment 1: ANOVA Results, Post-hoc Test
Running an ANOVA, we see that there is a significant effect of the treatment and on the trading error [F(3, 1544)=10.94, p < 0.1%] for the states with manipulation incentives. The pair-wise comparison shows the same results as the regression; participants in the treatments DETERMINISTIC and PROBABILISITC behave similar to each other but different to the treatments RANDOM and UNCONDITIONAL in states with manipulation incentives.
Behavioral Metric, Measuring the Extent of Manipulation in Experiment 1
As additional metric for manipulation, we analyze individual trading behavior by looking at the ratio of trades that participants do to drive the price towards the correct value. In order to detail the treatment differences we use linear regressions on the ratio of correct trades per market. As baseline we use the RANDOM treatment.
Table A2 gives the regression results. As expected we see that participants learn to improve their trading as with an increasing number of rounds played, their share of correct trades increases. As expected, in the RANDOM treatment there is no difference between the Manipulative Situations and the Non-Manipulative Situations as the dummy “Non-Manipulative Situations” is small and insignificant. If we look at the interaction effects, we see that in treatments DETERMINISITC and PROBABILISITC, users have a higher ratio of trades driving the price in the right direct in Non-manipulative Situations. As expected this is not the case in the UNCONDITIONAL treatment. We conclude that participants improved their trading skill and understanding of the task and that there are clear behavioral differences between treatments.
Table A2: OLS regression on the ratio of trades driving the price towards the correct value by treatment and situation for Experiment 1
Estimate / Std. Error(Intercept) / 0.58 / *** / 0.02
DETERMINISTIC / -0.08 / *** / 0.02
PROBABILISITC / -0.11 / *** / 0.02
UNCONDITIONAL / -0.05 / * / 0.02
Non-Manipulative Situations / 0.02 / 0.02
DETERMINISTIC*Non-manipulative Situations / 0.08 / * / 0.03
PROBABILISITC*Non-manipulative Situations / 0.08 / ** / 0.03
UNCONDITIONAL*Non-manipulative Situations / 0.04 / 0.04
Round / 0.01 / * / 0.00
Note: * significance at 5%, ** significance at 1%, and *** significance at 0.1%. 6,184 observations and an Adj-R2 of 0.013. A two-sided censored Tobit regression [0-1] yields the qualitatively same results.
Analyzing the timing of trades in Experiment 1 and Experiment 2
Experiment 1 did not impose a time constraint on participants. To illustrate the frequency of trades over time, we binned the number of trades according to their execution time relative to the start of the trading round. Figures A1 and A2 have the results. In experiment 1, we see that most participants were done trading after a short period. 94% of all trades happened in the first 60 seconds (98% in the first 120 seconds). In order to keep participants synchronized, we had to decide on fixed time frame for the trading periods in Experiment 2. In the test rounds participants had 120 seconds to get used to interface and experiment. Thereafter a trading round lasted 60 seconds. One clearly sees that trading drops off towards the end. Thus, time constraints seem not to play a major role in explaining participant behavior.
Figure A1: Timing of trades in Experiment 1.
Figure A2: Timing of trades in Experiment 2.
Participant Compensation
The return from manipulation exceeded the return from honest revelation on average. As we paid relative to performance per treatment there was a significant incentive to manipulate. However average payment per treatment remained the same.
Table A3: Average sum of all earning per treatment in Experiment 1.
Treatment / UNCONDITIONAL / DETERMINSTIC / PROBABILISTIC / RANDOMVirtual Currency / -9.44 / 1.97 / 0.53 / -6.13
Instructions for Experiment 1
Welcome
You will participate in an experiment about prediction and decision making. The experiment takes about 20 minutes.
Your payment will be $1.00 U.S. in base pay and a bonus. Your bonus depends on your performance. It ranges from $0 to $3.00, where the average bonus is $1.50. We encourage you to follow the instructions carefully, make good decisions, and earn as much money as possible.
You will only be paid if you complete the experiment in one pass (about 20 minutes) without interruption. If you can’t do so right now, please don’t start now but return later.
You can participate only once.
When you use the “backward” function of your browser or close your browser window, you will be excluded from the experiment without payment.
Read the instructions carefully, we will test your understanding. If you fail the test we cannot accept your HIT.
Instructions page 1 of 4 - Overview
Please follow these instructions carefully – you will have to demonstrate your understanding by answering some questions correctly.
You will participate in 8 rounds. Each round proceeds along the same 3 phases:
1. Information
2. Trading
3. Outcome
The scenario is the following: There are two bingo cages (labeled A and B), each holding black and white balls. There is a principal who in phase 3 decides to draw the ball either from bingo cage A or B, and a computer will draw a ball from that bingo cage.
In phase 1, you receive private information on the number of black and white balls. In phase 2, you can trade in both markets. In phase 3 the principal makes a decision for either bingo cage and a ball is drawn. Also the outcome of the draw will be revealed.
During the experiment you can earn e-dollars. In the end, your earnings are converted into U.S. dollars. The final bonuses will be between $0 and $3, with the average bonus set to $1.50. The exact conversation factor will be determined based on the performance of today's players. The bonus will be paid out by tomorrow.
The better trading success, the higher your earnings.
The rounds are independent of each other: Each round, the number of black and white balls per bingo cage and your information is independent of the previous rounds.
Instructions page 2 of 4 – Private information
The bingo cages hold 100 balls each, some are black, and the others are white.
Once a round begins, you will truthfully be told how many black and white balls are in each bingo cage.
At the end of each round in phase 3, a ball (black or white) is drawn by the computer.
Instructions page 3 of 4 – Trading
In the trading phase, you can trade tickets which are essentially bets on the probability of drawing a black ball from the bingo cage.
• You can choose whether to trade, when to trade, and how much to trade.
• You may change your mind about your predictions too.
Typically, a way to assess the value of a ticket is to evaluate the probability of the outcome associated with it. If one thinks there is an 81% chance of the ball being black, then a ticket is worth 1 e-dollar to him 81% of the time, and zero e-dollars 19% of the time. On average, it is worth 81% × 1 + 19% × 0 = 0.81 e-dollars to him. This is the expected payout, to maximize this, the best strategy is to buy tickets when the market price is below 0.81 e-dollars and sell tickets when the price is above 0.81 e-dollars.
The screenshot on the left shows the trading interface. At the top, you see your private information as on the previous screen. Below you see a price graph that is updated each time you make a trade.
At any time, there is a current price at which you can buy or sell tickets. The volume of a trade is always 5 tickets. You trade by clicking the “Buy (+5)” button or the “Sell (-5)” button. When one buys, the price rises; when one sells, the price falls.
The trading screen provides additional information:
• Your holdings: Each round you start with 0 units. You may not hold less than -50 units or more than 50 units.
• Cash: When you buy, you pay in cash; when you sell, you get cash. Your cash balance might be positive or negative.
• Expected Value: This is your expected payout which equals your expected return minus the initial cash endowment of 50 e-dollars.
TREATMENT DIFFERENCES: DETERMINISTIC, PROBABILISTIC, RANDOM
Instructions page 4 of 4 – Outcome
The screenshot on the left illustrates how you are informed on the outcome of a round. You receive a payment based on your performance in trading.
After trading closes, the principal makes a decision for either A or B.
PROBABILISTIC
The principal bases his decisions on the final price in each market: The higher a price is compared to the other, the more likely the respective bingo cage will be chosen. If the final price for ticket A is way higher than for ticket B, the principal will very likely (but not with certainty) decide to draw from bingo cage A. If the price for ticket B is higher than the price for A, the principal will likely decide for B. If final prices in both markets are equal, the decision is entirely random with equal probability for A and B.
DETERMINISTIC
The principal bases his decisions on the final price in each market: He chooses the bingo cage where the final market price is higher. If final prices in both markets are equal, the decision is entirely random with equal probability for A and B.
RANDOM
After trading closes, the principal makes a decision for either A or B. The decision is entirely random with equal probability for A and B.
In the example the decision is for market B. Thus, the value of your holdings of ticket B is determined. As the decision was for market B in this example, the trading in market A is not relevant for your bonus. Each ticket is worth 1 e-dollar if the ball is black and zero e-dollars otherwise. In the example, a ticket is worth 1 e-dollar and, thus, your total holdings of 15 tickets is worth 15 e-dollars. If your holdings were -10 tickets, and the ball was black that would be worth -10 e-dollars.
You received an initial endowment of 50 e-dollars per market. This endowment is now subtracted. The cash position is 41.13 and is added to the holdings (worth 15 e-dollars) which sums up to 6.13 e-dollars you earned in this round.
TREATMENT DIFFERENCES: UNCONDITIONAL