See first: Posterior Probability.
The Bayes rule allows us to compute the conditional probability P(X|Y) from P(Y|X):
(1)
From Tutorial on Bayesian nets and probability by Norman Fenton:
The power of Bayes' rule is that in many situations where we want to compute P(A|B) it turns out that it is difficult to do so directly, yet we might have direct information about P(B|A). Bayes' rule enables us to compute P(A|B) in terms of P(B|A)
For example, suppose that we are interested in diagnosing cancer in patients who visit a chest clinic.
Let A represent the event "Person has cancer". Let B represent the event "Person is a smoker".
We know the probability of the prior event P(A)=0.1 on the basis of past data (10% of patients entering the clinic turn out to have cancer). We want to compute the probability of the posterior event P(A|B). It is difficult to find this out directly. However, we are likely to know P(B) by considering the percentage of patients who smoke – suppose P(B)=0.5. We are also likely to know P(B|A) by checking from our record the proportion of smokers among those diagnosed. Suppose P(B|A)=0.8.
(2)
Thus, in the light of evidence that the person is a smoker we revise our prior probability from 0.1 to a posterior probability of 0.16. This is a significance increase, but it is still unlikely that the person has cancer.
Adapted from A brief introduction to Bayes' Rule by Kevin Murphy:
Here is a simple introduction to Bayes' rule from an article in the Economist (9/30/00). (my emphasis, Rei)
"The essence of the Bayesian approach is to provide a mathematical rule explaining how you should change your existing beliefs in the light of new evidence. In other words, it allows scientists to combine new data with their existing knowledge or expertise. The canonical example is to imagine that a precocious newborn observes his first sunset, and wonders whether the sun will rise again or not. He assigns equal prior probabilities to both possible outcomes, and represents this by placing one white and one black marble into a bag. The following day, when the sun rises, the child places another white marble in the bag. The probability that a marble plucked randomly from the bag will be white (ie, the child's degree of belief in future sunrises) has thus gone from a half to two-thirds. After sunrise the next day, the child adds another white marble, and the probability (and thus the degree of belief) goes from two-thirds to three-quarters. And so on. Gradually, the initial belief that the sun is just as likely as not to rise each morning is modified to become a near-certainty that the sun will always rise."
Mathematically, Bayes' rule states
(3)
or, in symbols:
(4)
where P(R=r|e) denotes the probability that random variable R has value r given evidence e. The denominator is just a normalizing constant that ensures the posterior adds up to 1; it can be computed by summing up the numerator over all possible values of R, i.e.
(5)
This is called the marginal likelihood (since we marginalize out over R), and gives the prior probability of the evidence.
