Scientific inference

First broadcast July 16, 2000.

Last year, I showed a group of young science students a mug with a spoon-like handle sticking out. From where they sat, they could see steam rising from the surface as I stood, holding the mug. Write down as many facts as you can about this, I told them, stirring the contents, so they could hear the sound it made.

Lulled by what they thought they knew of me, they wrote that the mug held hot coffee, and based on the stirring, they decided it contained a spoon and so the coffee was sweet. In fact, it was hot water with blue food colouring, and the "teaspoon" was a butter knife. When I revealed this, they took their first step in distinguishing inference from fact, but they weren't amused at being tricked that way. It wasn't funny, they exclaimed, it wasn't fair, they complained.

I don't see why, but I think my son's sense of humour is sadly deficient as well, and an exchange which occurred between us some years ago proves this. He'd had a small operation done a few hours earlier, nothing major, just a slight tweak to an eye muscle, but the surgeon had decided to use a general anaesthetic on him.

It seems his reaction to this sort of anaesthetic is to throw up rather violently, but I thought I'd try to make light of the whole matter. And that was how I came to be sitting at my son's bedside, discussing Pliny the Elder. It was, I suggested, less than surprising he'd reacted as he had, for Pliny had explained it all, long ago. Rather than paraphrase Pliny for you, let me share with you Philemon Holland's elegant version, as I did with him.

You'll quickly detect from the style that our translator of Pliny's Latin has been dead several centuries, but I think you can follow his version of what Pliny wrote about "The Veins of the Eyes". These veins, by the way, are what we would now call the optic nerves.

"Many right skilful masters in chirurgery, and the best learned anatomists, are of the opinion that the veins of the eyes reach to the brain. For mine own part, I would rather think that they pass into the stomach. This is certain, I never knew a man's eye plucked out of his head, but he fell to vomiting upon it, and the stomach cast up all within it."

As I say, I think my son's sense of humour was more than somewhat lacking. Or maybe it's his power of speech: I can't recall him speaking to me since then. Of course, Pliny, like my students, observed carefully. Like them, he went wrong in his inferring a link that simply wasn't there. It was an unusual false inference, for most of them are about causes of things, not neural pathways. Most, maybe, but not all of them, as this next example shows. It's an old tale, but the moral remains valid.

Once, there was a man who owned an exceedingly well-trained redback spider. He could say to the spider "Forward!", and the spider would move ahead. He could also tell his spider: "Back!", and the spider would go into reverse. Soon, the spider learned to go sideways on command, either to the right or to the left.

This red-back spider was a thing of pleasure to the man, who had trained other spiders before, but never to such a peak of perfection, and after long and arduous experiments on other spiders, he conceived a Grand Unified Theory of Spiders.

The man scurried off to a Famous Scientist with his best spider, to show her what he'd deduced. First he had the spider go forward, which the spider did, then backwards, then left, and then right. The Famous Scientist was most impressed, and said it was the best-trained spider she'd ever seen.

"Ah, but that's nothing," replied the man, and he produced a small pair of scissors. Four snips, and the spider was lopsided in the leg department. A few more snips and the spider was totally devoid of legs. Gently, the man laid his spider down on the table again, and addressed it as before. "Forward", he said, but the spider stayed where it was. "Back!", he cried, but the spider was motionless.

Then "Left!", he yelled, yet the spider remained fixed to the spot. Exultantly he shouted "Right!", but it was as if the spider had been glued there. The man turned triumphantly to the scientist, who was looking a little puzzled by this time. "You see," he told her, "Spiders hear through their legs!"

Now that sort of false inference we can all see through. If we couldn't, the joke would lose its point. Yet we let plenty of equally false inferences pass, unchallenged, almost every day, simply because we're too lazy to question what we're told, too lazy to question our assumptions.

Take Gauquelin, who gathered together the birth dates of many people, assigned them to star signs, and looked for evidence that certain star signs do better than others in certain sports and other activities. Now here I'd like to offer you the Law of the Wrongly Included Middle. In truth, it's just a variation on Ockham's Razor, but the Law of the Wrongly Included Middle offers a special viewpoint that I think allows it as a separate law. It goes something like this:

When you have some data and a set of results, stop including pet theories in the middle, at least unless you have a good reason to do so. In other words, Gauquelin had some data, the birth dates, and a set of results, the performances of the people selected. There was no good theoretical or other reason for including their star signs in the middle of the inference.

After all, no astrologer has ever claimed Cancers run faster sideways, or any other ways for that matter, or that Leos have a killer instinct in tennis, so where's the theoretical justification? No, what Gauquelin should have done, what his gullible audience ought to have done, was to stay with the outsides of the sequence, and see if they could explain the link from that. I certainly can.

Suppose you're a seven-year-old, halfway-good, tennis player. You play in a competition, you start winning against other seven-year-olds, and so you feel encouraged to play more, and to try even harder. Now a child born in January, while legally a seven-year-old for the purposes of the competition, may be almost a year older than some poor unfortunate December birthday seven-year-old.

All other things being equal, which kid will do better? I reckon young January will prove to have a head start, every time. And the same thing will happen in other sports, as well, though the cut-off dates may vary, so kids born in March or July may have the edge in football, or whatever.

And who's the juvenile tennis player most likely to persevere, to train hardest, to develop the winner's mental set? Young January, of course, with no need at all to refer to star signs, hocus pocus, the weight of the average Easter bunny, or even the ETA of the next inward-bound Tooth Fairy. All it took was a bit of logic, and the exclusion of the wrongly included middle.

Of course, the same sort of effect happens with plenty of other things as well, and sometimes, just sometimes, the inference is right, even when the logic is all wrong. Take the case of John Snow and the Broad Street pump, a favourite example of mine. Around 1853, Snow supposedly drew a map of cholera cases in a part of London, near Soho, where there were several cholera epidemics about this time.

Then, says the story, Snow noticed how cases seemed to cluster around the pump on a public well in Broad Street. I suspect he had a bit more than that to go on, and the map came later, but the standard version says he did it from the map, so we'll say so too. After all, who am I to spoil a good tale?

The next step, according to the yarn, is that Snow told the local health authorities there was a simple way to prevent the cholera epidemic in the district. Take the handle off the Broad Street pump, he said. Well, what is certain is that the handle was taken off the pump, the epidemic stopped, and nobody knew why. Germs, you see, hadn't been discovered yet.

Here we have a simple tale of Snow leaping to conclusions, making unsupported and unsupportable inferences, and being treated as a hero for it. In point of fact, there seems to have been a great deal more.

Snow had carried out careful enquiries, and knew more than what was on the map. He knew who had what water habits, so the map was just icing on the cake. What appears in the standard hagiographies as almost a random shot of mapping and going Aha! was nothing of the sort.

It seems almost as though one has to gamble, in the hope of getting it right. If you guess right, that's fine: if wrong, you drop into oblivion, but sometimes, your inference may carry on, long after you. Take the ancient Romans and their explanation of malaria. It always seemed to occur near swamps, and swamps smell rather awful, so it seemed reasonable to infer that swamp smells cause malaria. No sign of a cause, a bit like John Snow with the cholera, but no worries: drain the swamps, get rid of the smells, and presto, no more malaria! The inference seemed to stand up, at least as well as the former malaria victims did.

Now, of course, we know that draining the swamps got rid of the mosquitoes which spread the malaria, but it isn't easy, recognising the mosquitoes which carry malaria: you really need to count the hairs on their legs to tell the harmful and harmless species apart, and that needs a microscope. You also need a microscope to see the malarial bugs in the blood and in the mosquito, so this really slowed things down. On top of that, you needed the germ theory, to make people look in the right place, and that was a long time coming.

So for about two thousand years, the second-best bet, draining the swamps to get rid of the smells was in good odour, you might say, and it was even fairly successful. At least the act of draining stopped lots of people from getting malaria, even if the inference was all wrong, and there was no theoretical basis for blaming the smells of the swamp.

So you have two things: the observation, and the inference you make from it. It seems pretty clear to us that inferences have to be tested, but what about the observations? Should we test those as well, should we similarly call them into question?

Take the malaria example: the observation that swamps and malaria go together was quite clearly just that: an observation. But it was a limited observation, a flawed observation. Somebody should have gone around, looking at a few other swamps. Then they would have found that you didn't always get malaria, even if the smell was the same as that of a malarial swamp.

Yet until somebody makes the further observation, the original observation remains, and the equally flawed inference seems to have adequate support. How lucky for us that Ockham's Razor can save us.

Recently, I was wading down a shallow creek, in what I assumed was fresh water. After all, the map said I was still some ten metres above sea level, and that's rather more than the tidal range round my neck of the woods. All of a sudden, I noticed mangroves around me. Saltwater plants, those, or at least estuarine plants, which had no right to be in my fresh-water stream. Observation: mangroves. Inference: the water must be salty. Test: suck it and see.

Since I knew the creek's catchment area to be relatively unpolluted, I took the risk of becoming a martyr to the scientific method, along with the likes of Pliny the Elder and Francis Bacon. So I tasted the water, but I quickly spat it out again. It was salty, so I could rule out an error in botanical identification.

Further inference: Ockham's Razor says I can also rule out the possibility of somebody shovelling truck-loads of salt into the stream, so either I was wrong about where I was on the map, or the map was wrong. Now that's a good sort of inference: it exhausts the universe of possibilities, so one way or another, I have to be right, and I also have a testable proposition.

And that's the real point about science: inferences should never be placed on pedestals. They're working hypotheses, approximations to the truth. You can test them, in an emergency, you may well do a John Snow, and act on your inferences, but Heaven help you if you do any more than that. You may very well come unstuck.

Yet if people didn't infer from things, and test their inferences, we'd have very little science to play with. In fact, I believe one of the failings in modern science education may be our failure to develop students' inferential skills. But we must remember to keep our inferences in check, in proper perspective.

So next time you see a friend with a flashy car, go easy on those wild assumptions about lottery wins. Maybe your friend has borrowed the car. Or maybe, your friend has just joined the ranks of the car thieves.

This is one of a set of talks which were originally heard on ABC Radio National in Australia. All of the talks are copyright © Peter Macinnis, blank, but permission will be readily granted on request for educational and most non-profit purposes—I'm not particularly territorial, and on this one in particular, I'm generally delighted to share.

Contact Peter Macinnis specifying the talk(s) you want, and the purpose to which they will be put.

Warning: that email is a spam trap that gets checked at rare intervals. To get a prompt response, put my first name at the start: petermacinnis, not macinnis. I do this to mess up web-crawling robots that collect addresses--it's low-tech but too hard for them.

For the rest of the talks, go to Six Months of Sundays.