Arudra Burra

1 December 2007

Manipulation

Consider the following scenarios:[1]

  1. A supermarket sprays the smell of freshly baked bread on its premises, though no bread is baked there.
  2. Toy companies advertise a product heavily in the weeks leading up to Christmas, and then deliberately reduce the supply of the toy at Christmas-time. Since parents have by this time already promised to buy the advertised toy, they must buy it once the supply resumes in the new year – having already bought something else for their children at Christmas-time.
  3. A real estate company maintains a list of run-down houses at inflated prices, which it has no intention of selling. However, whenever an agent takes a client out to look at houses, he always shows them these houses first, so that the later ones look better in comparison.
  4. An influential economic policy-maker gains support for his (fairly right-wing) ideas by inviting economists from the loony right to his meetings: his own proposals, which seem modest and reasonable in comparison to theirs, are much more convincing as a result.
  5. Representatives of a “County Youth Counseling Program” are looking for college students to chaperon a group of juvenile delinquents on a day trip to the zoo. They begin by first asking for an even larger favor – namely to spend two hours a week as counselors to juvenile delinquents for a period of two years.[2]
  6. In trying to get a prisoner to confess to a crime, two police officers assume the Good Cop/Bad Cop routine – one of them is abusive, aggressive, and threatening; the other is friendlier, conciliatory, and tries to restrain his partner. The prisoner starts regarding him as a savior of sorts, and confides in him.
  7. A mortgage-lender offers you a loan with a low rate of interest to begin with, but which rises sharply after a few years. You don’t know much about the mathematics of compound interest (or you have some kind of bad discounting function), and sign up.
  8. A purposely does B a small, unsolicited favor, such as buying him a coke when he takes a break from the art appreciation class which both of you are attending. Later A asks B if he will purchase some raffle tickets which A is selling (the cost of the raffle ticket is more than the cost of the coke).[3]
  9. You are asked whether you have a commitment to helping the environment; as soon as you say yes, the person who asks you this asks if you will donate $5 to a foundation that supports this cause.
  10. Drug companies offer “lunch and learn” seminars to psychiatrists, to tell them about their latest drugs, and pay psychiatrists to run these seminars for them. All the participants involved know that these are run for commercial purposes, but both those who offer them, and those who attend them, become more favorably inclined to the drug in question than they would otherwise have been.[4]
  1. Initial observations
  1. These cases seem plausibly characterized as cases of manipulation. They cases strike me as morally suspect, though it’s difficult to say just why. But in these situations people at the receiving end would be aggrieved to learn about the mechanisms by which their agreement was secured, and I think rightly so. The problem seems to be that (in some obscure way), the mechanisms bypass the rational faculties of the people whose agreement is thus secured.
  2. Here are some other moral notions in the neighborhood. We feel that the manipulators have acted in bad faith, and used these people as a meremeans to an end.
  3. In many cases, one can (wrongfully) manipulate people without causing them any harm. For example, suppose someone forgets to add ‘bread’ to their shopping list before going to the supermarket: the smell of baked bread may be a welcome reminder. Or suppose the real-estate agent’s tactics actually help to correct for some cognitive irrationalities that would otherwise have been to her client’s detriment [AA].[5]
  4. Manipulation seems to depend constitutively on the intentions of the parties involved: when the supply of toys drops for no fault of the manufacturers’, I do not take them to have done any wrong whatsoever: the customers have simply no cause for resentment against them, though they may bemoan their bad luck.[6] (More generally, I wonder if the wrongdoing/blameworthiness distinction collapses in cases of harmless wrongdoing?)
  5. The wrong of manipulation also seems to depend on the purposes for which the agent is manipulated, and the effects of their being so manipulated. Many psychological experiments involve manipulation of just the sorts described above; but I am not tempted to regard them as morally suspect.[7] That’s partly because in these cases successful manipulation does not have any concrete consequences for the subject (e.g. parting with money to buy a car), and partly because manipulation was not intended to be a means of bringing about these consequences.
  6. One can manipulate people for good causes, such that, all things considered, manipulation is morally acceptable. Think of our reactions to some advertisements for charitable causes, e.g. to help famine victims in Africa. The strategy is often to get money by inducing guilt.
  7. Provisionally, one might say that manipulation seems to involve taking advantage of a vulnerability in order to get something from an agent – in a way that the agent would not endorse if (a) he was made aware of the vulnerability, or (b) he was made aware of the other’s motives.
  1. Further observations
  1. Our moral reactions are most naturally engaged when the manipulation involves some form of deception or misrepresentation – there it seems straightforward to assign manipulation to this general class of wrongdoing (the vulnerability consisting in such cases of some epistemic defect). But most of these cases are not like that: no representation is made at all, e.g. in the case of the toy company, or the economic policy-maker. Nor are these like straightforward cases of non-disclosure, e.g. when I don’t tell you about the termite-infestation in the house I am selling you.
  2. Many of these (and other) cases seem to involve taking advantage of common cognitive or motivational irrationalities, such as the tendency to trust people we like, or who are like us; our need to preserve consistency in our choices or to reciprocate favors; undue optimism in our abilities to withstand certain kinds of pressure or flattery; or the inability to do math (presumably any of the standard cognitive biases can be bases for manipulation – e.g. confirmation bias, optimism bias, anchoring, the gamblers fallacy, etc.).[8] Note that these characteristics need not be free-standing vulnerabilities – they may be highly valuable in general, just apt to go systematically wrong in some domains.[9]
  3. Sometimes success of manipulation seems to depend upon an agent’s being unaware of either (a) having the vulnerability in question, (b) the other’s motives, (c) the other’s purposes, or (d) the strategies being employed.[10] So in cases like the juvenile delinquency volunteer situation, or in cases of “low-balling” (where you induce an initial commitment, e.g. to buying a car, and only then mention a variety of extra costs), manipulation would not succeed if the manipulators strategy was transparent to the manipulatee. Books like Cialdini’s are written, in fact, in the hope that learning about these strategies will make people less susceptible to manipulation.
  4. But other cases are not like this. First of all, one might be aware of possessing a weakness, but not be aware that the other person is trying to take advantage of it (one happens let one’s guard down just that one time).
  5. Second, one might be aware of both the weakness and the other’s motives, but overestimate one’s own ability to resist. For instance, we tend to like people who praise us even when we know they are flattering us, and know that they are flattering us in order to get something out of us. So we may regard ourselves as immune from, e.g., biases towards physically attractive people.[11]
  6. Third, there are many biases which are difficult to overcome even once one has become aware of them, though one can try and overcompensate in the other direction. For instance, the instruction by the judge to “ignore what you’ve just heard” in a jury trial seems like it would be very hard to comply with. Also for the Good Cop/Bad Cop case.
  7. Fourth, the interests at stake may be such that we submit even though we know we are being manipulated. Some cases of “emotional blackmail” may be like this – e.g. when one is induced to do something because of deliberately induced guilt (say in the charity case). Or take the case of the rock-climber who refuses to buy protective gear, because he knows that his parents are so concerned for his safety that they would buy it from him. Or (to take a case in which the interest is very weak), children sometimes try to manipulate adults by using their “cuteness” to get something they want – and sometimes we give it despite knowing full well what they are up to.
  8. Finally, the strategy of manipulation may include passing on the fact that an attempt at manipulation is being made. E.g. “Of course, you know I’m trying to seduce you, don’t you?” or “Take all this with a pinch of salt – I’m only a salesman.”
  9. So manipulation may be (a) non-transparent, (b) transparent but resistible, or (c) transparent but irresistible. The closest analogy may be with quasi-addiction and hypnosis. If I put a drug in your coffee to make you more hungry, and then tell you about it, it maybe possible to resist the urge to eat, but then again perhaps not. Similarly, Anthony tells me, with the case of hypnosis – when he was hypnotized to eat a cookie (without being hypnotized to forget that he was so told), he was fully aware that the desire was in some sense “planted” in him, but he was unable to resist even though he very much wanted to.
  1. Manipulation and coercion
  1. I have been pushing the general line that coercion and fraud should be thought of as proceeding in two stages: creating a vulnerability (e.g. inducing fear or a false belief), and then taking advantage of it. Typically we confuse these two because the same person does both. But of course it is possible to take advantage of a vulnerability without having first caused it – these are cases of exploitation and non-disclosure respectively.
  2. Here is an easy way to see the distinction. I hire someone to threaten someone (for their money of their life). Then I come on the scene and offer to get rid of the threat for a fee. There seems to be nothing morally important that differentiates this case from the paradigm case of coercion (this is presumably how protection rackets are framed). [Actually, I’m not sure what the point of this example is, but I like it!]
  3. Because the paradigm case of coercion is the “gunman” case, we tend to associate coercion with violence, and we tend to assimilate the wrongfulness of coercion with the wrongfulness of the violence. This makes it hard to understand the phenomenon of “coercive offers” – e.g. an offer to prison inmates for some important freedoms in exchange for participation in a drug trial, or an offer to a give drugs to a heroin addict in exchange for something else. In fact, people who write about coercion are typically puzzled by this phenomenon.
  4. I think these cases are hard to analyze because we typically confuse what is achieved with the means by which it is achieved. To see this, consider the following three cases. Let us assume that A wants to get a large amount of money from B, who is an elderly, wealthy and somewhat lonely lady. There are three ways in which he might do this:
  5. Break into her house, and threaten her with physical violence unless she parts with the money;
  6. Put some sleeping pills in her soup, so that she doesn’t wake up when he enters to search the house;
  7. Ingratiate himself with her, pretend to fall in love with her, and induce her to do the same; then get her to transfer a certain sum of money to his bank account.
  8. It seems to me that only (a) would be described as coercive, but there is an important sense in which there is a wrong (admittedly difficult to characterize) which is the same in all three cases. Were A to deliberate about how to get the money from B, he might well be indifferent to how to go about doing it – and settle the question by various pragmatic considerations. This is not to say that each of these different strategies involve further and distinct wrongdoings (like marrying someone under false pretences, or subjecting them to threats of physical violence, etc.).
  9. My suggestion is that one ought to think of coercion by threats of force as a special case of manipulation: threats are merely one among many means of getting people to do something, and a fairly crude one at that. It is crude in two ways:
  10. They rely upon fairly crude vulnerabilities, such as the fear of pain or death;
  11. They succeed only by being transparent – in order to be coerced, one must know one is being coerced; this is not so with other forms of manipulation.
  12. Once one sees the central wrong as one involving manipulation or taking advantage of a vulnerability, one can see why there is a temptation to think of “offers you can’t refuse” as being somehow coercive. Harry Frankfurt’s response to the problem is to deny that the threat/offer distinction should be analyzed in terms of departures from some baseline. He points out that “threats appeal to motives and desires which are beyond the victim’s ability to control.”
  13. My point is that one can extend Frankfurt’s analysis further – there are many motives and desires which are in some sense “beyond the victim’s ability to control,” and which can be used to get something from the victim. These can include (as in the case above), the need to be loved, or the need to maintain a sense of honour. Threats only engage one sort of motive/desire.[12]
  14. It’s striking that people who talk about brainwashing or deceptive advertising often refer to the phenomenon as “coercive persuasion” – and I think this analysis – rather than analyses which rely on the notion of ‘consent’ helps explain why. It’s an easy response to think that the people who say this are just silly, because obviously coercion involves, threats, etc. But I think we end up assimilating many objectionable ways of doing things to cases of coercion, because we simply lack a broader vocabulary in which to talk about them; so it’s helpful to see that both coercion and manipulative persuasion, deception, etc. fall into a single moral category.
  15. In Manufacturing Consent, Noam Chomsky made the point that, in democracies, it is much easier to control people by, e.g., controlling the mass media, than using coercive means. In some ways (the claim has been elaborated), people in dictatorships can have a sense of integrity, because they know that they do things for fear of the secret police – a sense of integrity absent in (some democracies) because people think they do things because they want them. I think this analysis shows a way of fleshing out this kind of claim (and related claims about ‘ideology’), in a way that shows it’s not crazy on its face – though it may be false. I think they are right to think that the central notion is not violence or threats, but rather control and power.
  16. There might be something, too, to Plato’s fear of the poets and the demagogues – since what they do may not in fact be that different from what the tyrants do. [I shall have to look all this up – it would be nice if he also has an analogy with force].
  17. So I think that moral concerns which are typically discussed in the vocabulary of coercion and consent are better discussed in terms of manipulation, vulnerabilities, and ‘taking advantage’. In particular, we should think of both coercion and fraud as instances of a broader category of wrongdoing, which need not involve either threats of fraud, or the intentional inducement of false beliefs.

1

[1] Many of these examples are discussed and analyzed by Robert Cialdini in Influence: Science and Practice (Allyn and Bacon, fourth ed., 2001).

[2] This is from an experiment conducted by Cialdini et al. The number of people who agreed to take the juveniles to the zoo was three times greater in the condition in which the stronger request was made first than in the condition in which no such request was made.

[3] From an experiment by Denis Regan. Subjects who got the unsolicited coke bought twice as many raffle tickets than subjects in another group who didn’t get the coke.

[4] See “Drug Rep” by Dr. Daniel Carlat, in a recent piece in the New York Times. Daniel Rothschild made a similar point. [Also raise issue of the sale of prescription information to target sales]

[5] Note that the purpose of manipulation need not be to cause harm – so these are not cases in which manipulation has failed. (Of course, failed attempts at manipulation may fail to cause harm as well).

[6] Some tricky issues are raised by situations in which agents are involved – e.g. in the case of a pair of obtuse cops who are not trying to play particular roles, and are not even aware of the effect they are having on the prisoner. Imagine that their boss is perfectly aware of the effect they have when they work in tandem, and assigns them to work together precisely because of this effect.