StubbornFacts
Stubborn Facts
Stubborn Facts

Navigation

User login

Subscribe via RSS

Resources

The latest from our partner, the PoliGazette

Blog Roll

Your Moment in Probability

Submitted by Tully on Sun, 01/28/2007 - 2:13pm

We build a causal-link predictive model. We use 100% accurate and precise input data.

Assuming that your causal-link model is good enough to produce a correct predictive answer 99 out of 100 times, what are the expectations of the model output being correct after ONE cycle iteration? 99%, right? What about after TWENTY cycle iterations, sequentially using the output of each cycle iteration as the input of the next? FIFTY? After ONE HUNDRED cycle iterations?

Now try it at 95% probablility. And at 90%.

This simple thought experiment has some profound implications for the accuracy and certitude of real-world systems modeling, particularly extended complex models of organic systems that use meta-data inputs. Give it a try.

And this is why weather

And this is why weather forecasts are very accurate over the next hour, pretty good for the next 12 hours, alright for 24 hours, decent for tommorrow and absolute crap for next week.

Thanks for taking this brief moment to try and shovel back the tides of innumeracy that are drowning our civilization.

Yep

And in this hypothetical example ALL of your underlying assumptions are essentially correct, the system is both simple and "closed," and the only variance is due to the causal interactions not being known to infinite mathematical accuracy and perfection, giving you the 1% error factor.

Anyone else?

Class? Anyone? Anyone? Bueller? Bueller? Bueller?

medical tests

The example I like to use is drug or aids or other medical testing. For example, if most of the people tested are clean, then even if the false positive rate is low, then it can be the case that if a given result is positive, it's can actually be more likely to be a false positive than a true positive.

So if you get bad news on a medical test, consider a re-test.

I don't have time to cook up some hypothetical numbers to iliustrate this right now, but I think I may have some handy from my graduate project. I'll search when I have time.

OK

Pure hypothetical. Suppose you have a test for a disease that's 99% accurate. Suppose further that the rate of incidence of the disease is 1 in 100.

Suppose you test 10,000 people. From among these folks, 9900 are disease free, and 100 have the disease. From among the 9900 healthy, you get 9801 correct IDs of negative, and 99 false positives. From the 100 sick folks, you get 99 correct positives, and 1 incorrect negative.

That gives you a total of 99+99=198 folks who received a result of positive for the disease. Of these folks, only HALF actually have the disease. So even though the test's error rate is 99%, the error rate of the positive results is 50%, because of the low incidence of the disease. Aint that a bitch?

Not exponential growth, but one of my favorite moments in teaching probability.

exponential decay

Exponential decay...just think how many people would be bummed out if they understood this!

Just run the

Just run the numbers....

This one's a simple exponential, and your built-in calculator in Windoze can do it easily. I'll do a little rounding to keep the strings short.

0.991 = 0.99 = 99%

0.9920 = 0.818 = 81.8%

0.9950 = 0.605 = 60.5%

0.99100 = 0.366 = 36.6%

Even when you're absolutely positive about that 99% accuracy, at the end of 100 iterations your odds of still being correct in your iterated projection are barely better than 1 in 3. But most models aren't even remotely that accurate, so run the lower probability checks. (I'll leave out the first power, that being a bit obvious.) Here's at 95%:

0.9520 = 0.358 = 35.8%

0.9550 = 0.077 = 7.7%

0.95100 = 0.006 = 0.6%

You're down to that barely one in three by twenty iterations. At 90%? Nine-in-ten chance of being on target? Gimme those odds at the horse track! But in an iterative predictive model?

0.9020 = 0.122 = 12.2%

0.9050 = 0.005 = 0.5%

0.90100 = 0.0000027 = 0.00027%, or less than three out of ten million.

This is with a simple iterative predictive model of a closed system running at 99%--95%--90% accuracy. Next time you hear any prediction of anything running out much longer than a dozen cycles, remember this small example. As marghlar so rightly pointed out, this is why the weatherfolks get fuzzier and fuzzier as the time extends.

Your odds are actually slightly better than shown--there is always the slim chance that once the chain of correct answers is broken that a "wrong" answer in subsequent iterations can be right by sheer random accident, thus "resetting the clock," so to speak. But it's not the way to bet.

Recent comments

Advertisements
StubbornFacts.us does not endorse the content of any advertisement

Featured Movie

Syndicate

Syndicate content

Who's online

There are currently 0 users and 1 guest online.