Correlation is not causation. Really?

"Correlation is not causation." Is that a true statement?

Edward R. Tufte says, "In fact, probably the shortest true statement that can be made about causality and correlation is "Empirically observed covariance is a necessary but not sufficient condition for causality."

Edward R. Tufte, "The Cognitive Style of PowerPoint: Pitching Out Corrupts Within," second edition, January 2006.

Optimism on the Employment Front

Daniel Gross is at it again. One could argue that Mr. Gross's reasoning is flawed: he is extrapolating from recent trends to reach his conclusions. But he's tentative enough that we can give him that one. If he is right, then the underlying trend to re-employment may be stronger than the published figures indicate. After all, the figures keep being revised upward, don't they?

I believe that I previously flagged this article over at my Facebook account; if it seems familiar, maybe you saw it there.

In any event, I hope this article provides you and yours with some Christmas cheer.

Coincidence and Astrology

In my recent post, "Wittgensteins Everywhere," I noted the coincidence that I ran across a couple of references to Prince Wittgenstein, in two different books, on different topics, on the same day. I adumbrated (facetiously, I assure you) the possibility that there was some "Grand Plan" causing these apparently coincidental events in my life. A friend has since cautioned me about attributing events to "grand plans."

That's always a good caution. After all, as is well known, incompetence (greed, normal human foibles) is a better explanation of untoward events than conspiracy. (Yes, I think Lee Harvey Oswald acted alone. And, while I found Kevin Phillips' book American Theocracy interesting, I don't for a moment believe that there is a conscious, intentional, conspiracy underlying the events Phillips finds to be so suspicious.)

Moreover, if one has a sufficiently large sample, one will often find "strange coincidences" that are not strange at all. One of my favorites in the realm of statistics is that there was a perfect correlation between the price of rum and the salaries of Congregational ministers over quite a long period of time. Was there a secret compact between the rum dealers and the Congregational Church (which was, after all, the established church in Massachusetts)?

Not really. Over a very long period of time, prices have trended upwards. This is due to a number of things: rising populations, exhaustion of some resources, a constant tendency of governments to debase the currency, the discovery of gold and silver in the New World, and so on. Since prices of most goods and services are rising most of the time, all prices are positively correlated to some extents. (This is how your stockbroker can assure you that the stock market always, in the long run, rises. However, as Lord Keynes once said, in the long run we're all dead.) If one has a sufficiently large universe of prices, two of them will be perfectly correlated.

There is an old trick question illustrating this phenomenon. How many people do you have to have in a group before the chances of two of them sharing the same birthday go above 50%? Well, you might reason, there are 365 (and 1/4) days to choose from, so there would need to be 183 people in room before one of them has the same birthday I have. True, but that wasn't the question. The correct answer is, as I recall, 24. (Corrections and explanations are welcome.)

So, it doesn't really take all that many cases to have a sufficient universe for various "coincidences" to appear. If I ask, "Isn't it strange that two very different books happen to mention the same man?" - that looks like a coincidence. If, on the other other hand, I ask, "Out of the hundreds of men mentioned in the dozens of books I've read this year, wouldn't it be odd if some of them weren't mention in more than one book?" - then I'm looking at a predictable regularity.

Part of the fortune-teller's bit is to throw out so many vague predictions that some of them are bound to agree with some future event. But even more of it is due to our constant need to create patterns. When the patterns really do exist, as in the natural selection of biological traits, this human propensity is quite useful. When, however, the pattern isn't really there, one ends up with a specious theory. Astrology, for example, isn't just wrong because the stargazers can't agree on which predictions to make for which combinations of signs. Rather, it's wrong because it is based upon the positions of the constellations in the heavens. And the constellations do not exist. The stars really are randomly distributed, with respect to their visibility to a viewer on the Earth, so that any so-called "constellation" is purely a creation of the observer. Since the constellations don't correspond to any real pattern, they can't be predictive of events.

Unless, of course, you want to contend that the particular constellations we "see" are a result of some hard-wired characteristics of our minds - characteristics which also determine how we react to various circumstances. Sort of a celestial Rohrschach test.