Donald Rumsfeld made famous the saying that the absence of evidence is not evidence of absence. I don’t know anything about the origins of the saying, though I gather it was not original with Rumsfeld. The dictum itself may be unobjectionable, but Rumsfeld used it improperly and may have misled people thereby as to the meaning, and the limits, of this principle. I will explore here what the absence of evidence means.
The Absence of Elephants
You’re sitting at home on a Sunday afternoon, watching golf on television, which is to say that you are half-asleep, when the phone rings. The caller is one of your neighbors, and he says that there is an elephant in your front yard. You go to the window, you look out on the front yard, and there is no elephant there. You go back to the phone and say to your neighbor, “Good one. You made me look.” Then you hang up and go back to your golf.
I can divide your knowledge about elephants in your front yard into three states. In the first place, before the phone rings, you have no information. There is an absence of any evidence, as far as you know, regarding the presence or absence of elephants in your front yard. In fact, the possibility hasn’t even been raised. (It may be a general rule that a genuine absence of evidence coincides with a lack of interest in the question at hand.)
In the second state, after your neighbor calls, you have one piece of evidence: your neighbor’s testimony that an elephant is in your front yard. This may or may not seem likely to you, depending upon where you live and your prior experience. Under the principle of induction, if there has never been an elephant in your front yard before, it may seem unlikely that there is one at this time. However, the inductive problem, which indicates that any inductive conclusion can be overthrown by a single contrary example, implies that one should always be aware of the possibility of an instance contrary to one’s prior experience. One might call this keeping an “open mind.” Therefore, you go to the trouble of looking at the front yard, rather than dismissing your neighbor’s statement as a lame joke.
Finally, after you have looked at the front yard, you have another piece of evidence: your visual inspection of the yard shows that there are no elephants there. This evidence contradicts your neighbor’s statement, and (in your opinion) is conclusive. If you don’t see an elephant, there isn’t one. Your conclusion is that there is no elephant in your front yard.What I want to make clear with this example is that the absence of an elephant is not an absence of evidence concerning elephants. You have evidence concerning elephants, and this evidence is negative. That is, you have evidence of the absence of elephants. More generally, the absence of a phenomenon is not the same as the absence of evidence relevant to the existence of that phenomenon. Negative evidence is not the same as the absence of evidence.
The Absence of Elephants
You’re sitting at home on a Sunday afternoon, watching golf on television, which is to say that you are half-asleep, when the phone rings. The caller is one of your neighbors, and he says that there is an elephant in your front yard. You go to the window, you look out on the front yard, and there is no elephant there. You go back to the phone and say to your neighbor, “Good one. You made me look.” Then you hang up and go back to your golf.
I can divide your knowledge about elephants in your front yard into three states. In the first place, before the phone rings, you have no information. There is an absence of any evidence, as far as you know, regarding the presence or absence of elephants in your front yard. In fact, the possibility hasn’t even been raised. (It may be a general rule that a genuine absence of evidence coincides with a lack of interest in the question at hand.)
In the second state, after your neighbor calls, you have one piece of evidence: your neighbor’s testimony that an elephant is in your front yard. This may or may not seem likely to you, depending upon where you live and your prior experience. Under the principle of induction, if there has never been an elephant in your front yard before, it may seem unlikely that there is one at this time. However, the inductive problem, which indicates that any inductive conclusion can be overthrown by a single contrary example, implies that one should always be aware of the possibility of an instance contrary to one’s prior experience. One might call this keeping an “open mind.” Therefore, you go to the trouble of looking at the front yard, rather than dismissing your neighbor’s statement as a lame joke.
Finally, after you have looked at the front yard, you have another piece of evidence: your visual inspection of the yard shows that there are no elephants there. This evidence contradicts your neighbor’s statement, and (in your opinion) is conclusive. If you don’t see an elephant, there isn’t one. Your conclusion is that there is no elephant in your front yard.What I want to make clear with this example is that the absence of an elephant is not an absence of evidence concerning elephants. You have evidence concerning elephants, and this evidence is negative. That is, you have evidence of the absence of elephants. More generally, the absence of a phenomenon is not the same as the absence of evidence relevant to the existence of that phenomenon. Negative evidence is not the same as the absence of evidence.
The Case of Iraq’s Weapon’s Programs
Where Mr. Rumsfeld went wrong was in identifying an absence of material showing a particular outcome with an absence of evidence about that outcome. In fact, due to United Nations’ inspections, U.S. intelligence, and Iraqi reports, there was a lot of evidence about weapons of mass destruction in Iraq. On the one side, you had statements from Saddam Hussein and some of his underlings affirming that some weapons programs were in operation. (Whether they believed their own statements, or were running a bluff, is still uncertain.) On the other side, you had on-site inspections of places where such activities had taken place in the past, as well as places identified as suspicious by intelligence agencies, all of which showed that Iraq had complied with orders to terminate these programs.
In other words, the “elephant” in this case were weapons of mass destruction (WMD), and programs to manufacture and distribute them. Saddam Hussein himself, and certain intelligence sources claimed that there were active WMD in Iraq. On-site inspection, documentation, and a variety of intelligence sources contradicted those claims. Some photographic intelligence, much of which was (and always is) inherently ambiguous, supported the affirmative case. Other evidence supported the negative.
However you might wish to characterize the situation in Iraq in 2002 and 2003, an absence of evidence was not the problem. The problem was that the evidence available did not all point in one direction. It was not dispositive. It was clear that some of the weapons programs that had been in operation prior to 1991 were no longer functioning, at least not at the same scale or in the same places as they had been. It was clear that a lot of material had been destroyed under the sanctions regime, while there were no findings to show that it had been replaced. It also appeared that Saddam Hussein had the intention, or at least the wish, to develop WMD, if he ever had a free hand to do so.
In the end, by deciding to justify the invasion of Iraq on the existence of WMD, the Bush administration asserted that the affirmative evidence was more compelling than the negative evidence. A lot of people think they were wrong. To the extent that they dismissed the negative evidence as “an absence of evidence,” they were certainly wrong.
Conclusion
The absence of evidence means that one has no information about a subject. The absence of proof that something exists is not the same thing as the absence of evidence regarding its existence. Once one has evidence, one must evaluate it. Often, one needs to seek additional information to determine whether the initial evidence is credible or not. In many cases, the evidence is ambiguous or contradictory. Frequently, as in Iraq, the problem is not an absence of evidence: it is an inability rightly to assess the evidence one has.
2 comments:
"absence of evidence is not evidence of absence" is a general maxim in the historical sciences - archeology, anthropology etc.
In other words, you can't make the claim that such and such did not occur simply because you haven't found evidence of it yet. Of course, you also cannot claim it did occur.
The insinuation that Donald Rumsfeld made was that since no evidence could be found that proved positively that the WMD's had been destroyed, therefore they must still be there. Really poor evidence.
However, there is a general rule in signal theory that the only way not to have false positives is to reject everything and the only way not to have false negatives is to accept everything. It is always a balancing game - what percent confidence do we want to have in a decision? Invading Iraq had a very high confidence that they would be prevented from using any WMD's. Not invading Iraq would have had a lower confidence. Turn up the paranoia meter too high and we might as well nuke everyone first - that way no one will nuke us.
In a Democracy, you must have sufficient political will to successfully prosecute a war. Violence is only a successful means of political persuasion when it is overwhelming and crushing. Anything less than that and your enemy will keep on coming back over and over again. As a practical policy, unless your enemy has done something that really pisses off the citizens enough so that they will support your using overwhelming and crushing force, it is better to stick to the negotiating table.
Sean, you really need to work on the organization of your comments. For another thing, it would be really nice to have some authority for the acceptance of this saying as a maxim in the historical sciences. I spent a lot of time in history, political science, and economics classes over the years, and I never heard anyone say that the "absence of evidence is not evidence of absence." In fact, if there is a complete lack of evidence on a subject, why would the hypothesis ever arise?
The maxim I heard most often, especially in economics, was ceteris paribus - all other things being equal.
The more interesting point in your comment is about the type A and type B errors. Generally speaking, the social sciences like to have a .99 confidence level before asserting that something is true. That is usually interpreted as meaning that there is only a .01 chance that the given circumstance or correlation could have arisen by chance.
The other common practice is the use of the negative hypothesis, which parallels to reductio ad absurdum form of argument in deductive logic. Say one wishes to test the hypothesis that crime is caused by poverty. One would set up an hypothesis that crime was not caused by poverty, and then test it to see if it works. If you get a high confidence level in the negative hypothesis, you reject the original hypothesis.
One of the interesting facets of the Iraq war is that you are exactly wrong about the likelihood of Iraq using WMD. The highest likelihood of Iraq using chemical or biological weapons against us, if they had them, was precisely in the event that we invaded them. That they didn't use such weapons against people in the first days of the invasion was pretty good evidence right there that the weapons simply did not exist.
That is, they were absent in a situation in which you would have expected them to be present, if possible. The absence of use was evidence of their absence.
Finally, there are lots of cases in which the use of military force can be politically effective without destroying the opposing side. Look at the American Revolutionary War for a good example.
Post a Comment