Probability Models
A probability model is a mathematical representation of a random phenomenon. It is defined by its sample space, events within the sample space, and probabilities associated with each event.
The sample space S for a probability model is the set of all possible outcomes.
For example, suppose there are 5 marbles in a bowl. One is red, one is blue, one is yellow, one is green, and one is purple. If one marble is to be picked at random from the bowl, the sample space possible outcomes S = {red, blue, yellow, green, purple}. If 3 of the marbles are red and 2 are blue, then the sample space S = {red, blue}, since only two possible color outcomes be possible. If, instead, two marbles are picked from a bowl with 3 red marbles and 2 blue marbles, then the sample space S = {(2 red), (2 blue), (1 red and 1 blue)}, the set of all possible outcomes.
An event A is a subset of the sample space S.
Suppose there are 3 red marbles and 2 blue marbles in a bowl. If an individual picks three marbles, one at a time, from the bowl, the event "pick 2 red marbles" can be achieved in 3 ways, so the set of outcomes A = {(red,red, blue),(red, blue,red), (blue, red,red)}. The sample space for picking three marbles, one at a time, is all of the possible ordered combinations of three marbles, S = {(red, red, red), (red, red, blue), (red, blue, red), (blue, red, red), (blue, blue, red), (blue, red, blue), (red, blue, blue)}. Since there are only 2 blue marbles, it is impossible to achieve the event {blue, blue,blue}.
A probability is a numerical value assigned to a given event A. The probability of an event is written P(A), and describes the long-run relative frequency of the event. The first two basic rules of probability are the following:
Rule 1: Any probability P(A) is a number between 0 and 1 (0 P(A) 1).
Rule 2: The probability of the sample space S is equal to 1 (P(S) = 1).
Suppose five marbles, each of a different color, are placed in a bowl. The sample space for choosing one marble, from above, is S = {red, blue, yellow, green, purple}. Since one of these must be selected, the probability of choosing any marble is equal to the probability of the sample space S = 1. Suppose the event of interest is choosing the purple marble, A = {purple}. If it is equally likely that any one marble will be selected, then the probability of choosing the purple marble, P(A) = 1/5. In general, the following formula describes the calculation of probabilities for equally likely outcomes:
If there are k possible outcomes for a phenomenon and each is equally likely, then each individual outcome has probability 1/k.
The probability of any event A is
If two events have no outcomes in common, then they are called disjoint. For example, the possible outcomes of picking a single marble are disjoint: only one color is possible on each pick. The addition of probabilities for disjoint events is the third basic rule of probability:
Rule 3: If two events A and B are disjoint, then the probability of either event is the sum of the probabilities of the two events:
P(A or B) = P(A) + P(B).
The chance of any (one or more) of two or more events occurring is called the union of the events. The probability of the union of disjoint events is the sum of their individual probabilities.
For example, the probability of drawing either a purple, red, or green marble from a bowl of five differently colored marbles is the sum of the probabilities of drawing any of these marbles: 1/5 + 1/5 + 1/5 = 3/5.
If there are three red marbles and two blue marbles, then the probability of drawing any red marble is the number of outcomes in the event " red," which is equal to three, divided by the total number of outcomes, 5, or 3/5 = 0.6. The sample space is this case is {red, blue}, which must have total probability equal to 1, so the probability of drawing a blue marble is equal to 2/5 = 0.4. The event of drawing a blue marble does not occur if a red marble is chosen, so the event A = "blue" is called the complement Ac of the event "red." Since an event and its complement together form the entire sample space S, the probability of an event A is equal to the probability of the sample space S, minus the probability of Ac, as follows:
Rule 4: The probability that any event A does not occur is P(Ac) = 1 - P(A).
Venn Diagrams
A useful graphical tool for studying the complements, intersections, and unions of events within a sample space S is known as a Venn diagram. In such a diagram, events are drawn as regions that may or may not overlap.
In the Venn diagram to the right, events A and B are disjoint. Suppose, for example, event A is drawing a red marble from a bowl of five differently colored marbles, and event B is drawing a blue marble. These events cannot both occur, so there is no overlapping area.
In the Venn diagram on the left, events A and B are not disjoint. This means that it is possible for both events to occur, and the overlapping area represents this possibility. Suppose, for instance, there are 3 red marbles and two blue marbles in a bowl. Two marbles are to be drawn from the bowl, one after the other. After the first draw, the marble drawn is returned to the bowl. Define event A to be drawing a red marble from the bowl on the first draw, and define event B to be drawing a blue marble on the second draw. The occurence of event A is represented by the red area, the occurence of event B is represented by the blue area, the occurence of both events is represented by the overlapping area (also known as the intersection of the two events), and the occurence of either event is represented by the entire colored area (also known as the union of the two events).
Independence
Consider two events which might occur in succession, such as two flips of a coin. If the outcome of the first event has no effect on the probability of the second event, then the two events are called independent. For two coin flips, the probability of getting a "head" on either flip is 1/2, regardless of the result of the other flip. The fourth basic rule of probability is known as the multiplication rule, and applies only to independent events:
Rule 5: If two events A and B are independent, then the probability of both events is the product of the probabilities for each event:
P(A and B) = P(A)P(B).
The chance of all of two or more events occurring is called the intersection of events. For independent events, the probability of the intersection of two or more events is the product of the probabilities.
In the case of two coin flips, for example, the probability of observing two heads is 1/2*1/2 = 1/4. Similarly, the probability of observing four heads on four coin flips is 1/2*1/2*1/2*1/2 = 1/16.
If two events A and B are not disjoint, then the probability of their union (the event that A or B occurs) is equal to the sum of their probabilities minus the sum of their intersection.
In the example corresponding to the second Venn diagram above, we know that the probability of drawing a red marble on the first draw (event A) is equal to 3/5, and the probability of drawing a blue marble on the second draw (event B) is equal to 2/5. Since events A and B are independent, the probability of the intersection of A and B, P(A and B), equals the product P(A)*P(B) = 3/5*2/5 = 6/25. The probability of the union of A and B, P(A or B), is equal to
P(A) + P(B) - P(A and B) = 3/5 + 2/5 - 6/25 = 1 - 6/25 = 19/25 = 0.76.
For another example, consider tossing two coins. The probability of a head on any toss is equal to 1/2. Since the tosses are independent, the probability of a head on both tosses (the intersection) is equal to 1/2*1/2 = 1/4. The probability of a head on either toss (the union) is equal to the sum of the probabilities of a head on each toss minus the probability of the intersection, 1/2 + 1/2 - 1/4 = 3/4.
Note: Disjoint events are not independent. In the marble example, consider drawing one marble from the bowl of five, where each marble is a different color. Suppose the event of interest, event A, is drawing a blue marble. The probability of drawing this marble is 1/5. Suppose event B is drawing a green marble. These events are disjoint, since event B cannot occur if event A occurs. Obviously, they are not independent, since the outcome of event A directly affects the outcome of event B. If, instead, two marbles were to be drawn from the bowl, with the first marble replaced before the second marble was drawn, then the event of drawing a blue marble on the first draw would not affect the outcome of the second draw. The event of drawing a green marble on the second draw would be independent of the event of drawing a blue marble on the first draw, so the probability of both events occurring would be the product of the probabilities of each event, 1/5*1/5 = 1/25.
The probability of the intersection of two or events which are not independent is determined using conditional probabilities.
For some more definitions and examples, see the probability index in Valerie J. Easton and John H. McColl's Statistics Glossary v1.1.