Let's suppose there are 250,000 MS patients in the US, 125,000 of whom will go into remission this year. Suppose you are a high powered (and high paid, and while we're fantasizing about this why don't we imagine the red Porsche convertible and the vacation home on the beach) research scientist who wants to test a new MS medication. You advertise for MS patients who are not currently on medication to participate in your study. These people must agree to stay in the study for a year, to be examined once a month, and run the risk of making their MS worse or suffering horrendous side-effects in the name of Science. A total of 28 people apply and are accepted. Click on the "Sample MS Patients" below to discover what would happen to them if they go totally medication-free. Each time you mash on the button you will get a different group of 28 patients. So try it a few times.
Now suppose you are testing a new drug called Wormycin extracted from mashed up worms (the testimonial letters in Alternative Medicine must have turned your head). In reality it doesn't help MS at all, but you don't know this. (Nobody knows it yet. That's why you're doing the experiments, to find out how well it works.) This means that in reality if all 250,000 MS patients in the US took Wormycin, 50% of them - that's 125,000 people - would go into remission, the same as if they took no drug at all. So in the best scientific tradition you perform a double-blind test: you get 56 patients and separate them into two groups of 28. The first group gets given Wormycin for a year. The second group are given dummy pills. Neither group knows whether it is getting the real Wormycin or not. Then you measure how many patients went into remission. Mash on the "Run Wormycin Experiment" button to run the experiment for a year. Mash it several times to see the wildly different results that you get. In real life the experiment only gets done once, of course. So how's it going to turn out? Like the first time you mashed the button? The second? The third?
Now suppose you are testing a new drug called Antisclerosin (I'm making this up as I go along) that puts 75% of MS patients into remission, but you don't know that yet. This means that if all 250,000 MS patients in the US took Antisclerocin, 75% of them - that's 187,500 people - would go into remission. But you only get to try it on 28 patients. Mash on the "Run Antisclerosin Experiment" button to run the experiment for a year. Notice that if you mash it several times, in general you see higher numbers more often than you do when you mash the "Run Wormycin Experiment" button several times.
Now mash on the "Run Wormycin Experiment" until you see a percentage in the "Conclusion" box beneath it that looks particularly high. Then mash on the "Run Antisclerosin Experiment" until you see a rate that's lower than that. It should only take you a couple of dozen clicks on each button.
Remember that you only get one shot at it in real life. So what if the one time you do these experiments, you get these results? It would look like worms (which don't help MS) cure MS better than the drug (which would put an extra 62,500 MS sufferers into remission nationwide). Does that suck or what? That's the risk you run when you do scientific studies with a small number of patients. And it happens all the time. Remember, that's statistics.
Fact: Statistical tests on a small number of MS patients can give wild results.
Why am I telling you about this? I once read a report in a newspaper that described new research proving a correlation between MS and something else - I forget what it was, something that seemed totally unrelated. As usual, the article seemed to have garbled the science a little bit. What tipped me off was that the numbers didn't seem quite right. When you multiplied it out, it seemed to indicate that there should be a whole lot more MS sufferers than there actually seem to be. I asked my doctor about it the next time I saw him. As I will discuss later in The Medical Profession, I am fortunate enough to have found a doctor who is happy to answer any question I have about medicine, foolish or otherwise. He told me that the study was done with only 28 MS patients. As we can see from mashing on the buttons on this page and the previous one, that means that the results are probably not accurate.