Lab: Probability

CSC261 - Artificial Intelligence - Weinman

Summary:: We explore the application of logic and probability.

Preparation

Get out a sheet of paper (yes, paper) and something to write with.

Exercises

A: Plausible Reasoning

The inference rule modus ponens allows us to make the following deduction

A⇒ B,A

The strength of propositional logic lies in its ability, given a sound and complete inference algorithm, to determine what we may conclude from our background information. However modern AI systems deal with incomplete information and are still required to make decisions and take action. Thus, probability affords us a useful way of quantifying uncertainty about certain proposition and measuring the change in uncertainty as more observations are made.

In this exercise we will look at a weaker, probabilistic version of the modus ponens rule. Given the implication above, what happens to our belief about A when B is observed rather than A? Our intuition suggests that observing B, which is a consquence of A, increases our belief in A. Can we prove it?

Let C ≡ A⇒ B be our background information or knowledge. Write a probability equation relating A and B, conditioned on the background C that signifies the equivalence of reasoning about A first and then B (as in modus ponens) or else B first and then A (as we are now trying to do).
Using the rules of probability we have derived so far, find any terms in the equation above that may be simplified.
If you note that the background information you are given does not entail B, what can you say (numerically) about your belief in B?
Using this information, how would the observation of B influence your degree of belief about A? (That is, compare the probabilities before and after observing B).
Be prepared to walk through and share your answers with the rest of the class.

B. Bayes Rule

Test effectiveness is a big, real world issue. See the very recent controversy over ditching routine prostate cancer screening:

The assumption is that finding cancer early is always a good thing. Not so, said Dr. Virginia Moyer of the Baylor College of Medicine, who heads the [government] task force.
[The test] only sometimes signals prostate cancer is brewing. It also can mean a benign enlarged prostate or an infection. Worse, screening often detects small tumors that will prove too slow-growing to be deadly. And there's no sure way to tell in advance who needs aggressive therapy.
The task force analyzed ... whether routine screening reduces deaths from prostate cancer. The conclusion: There's little if any mortality benefit.
But there is harm from routine screening: impotence, incontinence, infections, even death that can come from the biopsies, surgery and radiation, Moyer said.
Associated Press, "Government panel recommends against routine PSA screening for prostate cancer," Washington Post, Thursday, 6 October 2011, http://www.washingtonpost.com/national/health-science/reports-say-government-to-recommend-against-routine-psa-screening-for-prostate-cancer/2011/10/06/gIQAPKCWRL_story.html

How might such a decision be made? Probably with a process not entirely unlike the following.¹ (All numbers and test below are entirely fictitious.)

Consider two medical tests, A and B for a particular type of cancer. Test A is 95% accurate at recognizing the cancer when it is present, but has a 10% false positive rate (indicating cancer is present when it is not). Test B is only 90% effective at detecting the cancer, and has a 5% false positive rate. The two tests use independent detection methods. Just one percent of men actually suffer from this cancer.

Suppose a person is tested for the cancer using only one of the two possible tests, and that test comes back positive. Which test returning positive is more indicative of someone actually having the cancer?

Define all the propositions that will be used to answer the question. For example

A ≡ Test A signals positive for the cancer.
Using these propositional symbols, write down all of the probabilities given in the problem statement above.
What probabilities must be calculated to answer the question "which positive test is more indicative?"?
Using the rules of probability, calculate the values needed. Write out intermediate probability terms before replacing with any numeric counterparts.
Which test returning positive is more indicative of someone actually having the cancer?
How much more likely is someone to have cancer if they test positive with the more effective test than someone who tested positive with the less effective test? (That is, calculate a ratio.)
Be prepared to walk through and share your answers with the rest of the class.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Footnotes:

¹Adapted from Russell and Norvig, 2010, Exercise 13.13.