The Goodness-of-Fit Test CK12 Editor
Say Thanks to the Authors Click http://www.ck12.org/saythanks (No sign in required)
To access a customizable version of this book, as well as other interactive content, visit www.ck12.org
CK-12 Foundation is a non-profit organization with a mission to reduce the cost of textbook materials for the K-12 market both in the U.S. and worldwide. Using an open-content, web-based collaborative model termed the FlexBook®, CK-12 intends to pioneer the generation and distribution of high-quality educational content that will serve both as core text as well as provide an adaptive environment for learning, powered through the FlexBook Platform®. Copyright © 2013 CK-12 Foundation, www.ck12.org The names “CK-12” and “CK12” and associated logos and the terms “FlexBook®” and “FlexBook Platform®” (collectively “CK-12 Marks”) are trademarks and service marks of CK-12 Foundation and are protected by federal, state, and international laws. Any form of reproduction of this book in any format or medium, in whole or in sections must include the referral attribution link http://www.ck12.org/saythanks (placed in a visible location) in addition to the following terms. Except as otherwise noted, all CK-12 Content (including CK-12 Curriculum Material) is made available to Users in accordance with the Creative Commons Attribution/NonCommercial/Share Alike 3.0 Unported (CC BY-NC-SA) License (http://creativecommons.org/licenses/by-nc-sa/3.0/), as amended and updated by Creative Commons from time to time (the “CC License”), which is incorporated herein by this reference. Complete terms can be found at http://www.ck12.org/terms. Printed: April 7, 2013
AUTHOR CK12 Editor
www.ck12.org
C ONCEPT
Concept 1. The Goodness-of-Fit Test
1
The Goodness-of-Fit Test
Learning Objectives • • • •
Understand the difference between the Chi-Square distribution and the Student’s t-distribution. Identify the conditions which must be satisfied when using the Chi-Square test. Understand the features of experiments that allow Goodness-of-Fit tests to be used. Evaluate an hypothesis using the Goodness-of-Fit test.
Introduction In previous lessons, we learned that there are several different tests that we can use to analyze data and test hypotheses. The type of test that we choose depends on the data available and what question we are trying to answer. For example: • We analyze simple descriptive statistics such as the mean, median, mode and standard deviation to give us an idea of the distribution and to remove outliers, if necessary; • We calculate probabilities to determine the likelihood of something happening; and • We use regression analysis to examine the relationship between two or more continuous variables. But what test do we run if we are trying to examine patterns between distinct categories such as gender, political candidates, locations or preferences? To analyze patterns like these we use the Chi-Square test. The Chi-Square test is a statistical test used to examine patterns in distinct or categorical variables, which we learned about in the earlier chapter entitled Planning and Conducting an Experiment or Study. This test is used in: 1. Estimating how closely a sample matches the expected distribution (also known as the Goodness-of-Fit test) and 2. Estimating if two random variables are independent of one another (also known as the Test of Independence - see Chapter 9). In this lesson we will learn more about the Goodness-of-Fit test and how to create and evaluate hypotheses using this test.
The Chi-Square Distribution The Chi-Square Goodness-of-Fit test is used to compare the observed values of a categorical variable with the expected values of that same variable. For example, we would use this test to analyze surveys that contained categorical variables (for example, gender, city of origin, or locations that people preferred to visit on vacation) to determine if there are in fact relationships between certain items. Example: We would use the Chi-Square Goodness-of-Fit test to evaluate if there was a preference in the types of lunch that 11th grade students bought in the cafeteria. For this type of comparison it helps to make a table to visualize the problem. We could construct the following table to compare the observed and expected values. Research Question: Do 11th grade students prefer a certain type of lunch? 1
www.ck12.org Using a sample of 11th grade students, we recorded the following information:
2
www.ck12.org
Concept 1. The Goodness-of-Fit Test
TABLE 1.1: Frequency of Type of School Lunch Chosen by Students Type of Lunch Salad Sub Sandwich Daily Special Brought Own Lunch
Observed Frequency 21 29 14 36
Expected Frequency 25 25 25 25
If there is no difference in which type of lunch is preferred, we would expect the students to prefer each type of lunch equally. To calculate the expected frequency of each category as if school lunch preferences were distributed equally, we divide the number of observations by the number of categories. Since there are 100 observations and 4 categories, the expected frequency of each category is 100/4 or 25. The value that indicates the comparison between the observed and expected frequency is called the Chi-Square statistic. The idea is that if the observed frequency is close to the expected frequency, then the Chi-Square statistic will be small. Or, if the difference between the two frequencies is big, then we expect the Chi-Square statistic to be large. To calculate the Chi-Square statistic (X 2 ), we use the formula: 2
i) X 2 = ∑i (Oi −E where: Ei
X 2 = Chi-Square statistical value Oi = observed frequency value for each event Ei = expected frequency value for each event Once calculated, we take this Chi-Square value along with the degrees of freedom (this will be discussed later) and look up the Chi-Square value on a standard Chi-Square distribution table. The Chi-Square distribution allows us to determine the probability that a sample fits an expected pattern. In contrast, the t-distribution tests how likely it is that the means of two different samples will differ. Please see the table below for more details.
TABLE 1.2:
The Difference Between the Chi-Square and the Student’s t-test when Using to Compare Two Sample Means
Type of Distribution
Tells Us
Every Day Example
Chi-Square
The relationship between two or more categorical variables.
Analyzing survey data with categorical variables.
Student’s t-Test
The differences between the means of two groups with respect to a continuous variable.
Determining if there is a difference in the mean of the SAT scores between schools.
Data Needed to Determine Value Observed and expected frequencies for categorical variables, degrees of freedom. The mean values for samples from two populations, degrees of freedom.
Features of the Goodness-of-Fit Test As mentioned, the Goodness-of-Fit test is used to determine patterns of distinct or categorical variables. As we learned in Lesson 6, a categorical variable is one that is not continuous and has observations in separate categories. Examples of categorical variables include: -gender (male or female) 3
www.ck12.org -preferences (agreed, neutral or disagreed) -behaviors (got sent to the office or didn’t get sent to the office) -physical traits (straight, wavy or curly hair) Categorical variables are not the same as measurement or continuous variables. The following are normally not categorical variables:
− height
− distance
− weight
− income
− test scores It is important to note that most of these continuous variables could in fact be converted to a categorical variable. For example, you could create a categorical variable with two values such as ¨Less that 10 miles¨ and ¨Greater than 10 miles.¨ In addition to categorical variables, a Goodness-of-Fit test also requires: -data obtained through a random sample -a calculation of the Chi-Square statistic using the formula explained in the last section -the calculation of the Degrees of Freedom. For a Chi-Square test, the Degrees of Freedom are equal to the number of categories minus one or d f = c − 1 Using our example about the preferences of types of school lunches, we calculate the d f = 3.
df = of categories − 1 3 = 4−1 There are many situations that use the Goodness-of-Fit test, including surveys, taste tests and analysis of behaviors. Interestingly, Goodness-of-Fit tests are also used in casinos to determine if there is cheating in games of chance such as cards and dice. For example, if a certain card or number on a die shows up more than expected (a high observed frequency compared to the expected frequency), officials use the Goodness-of-Fit test to determine the likelihood that the player may be cheating or the game may not be fair. Evaluating Hypothesis Using the Goodness-of-Fit Test Let’s use our original example to create and test a hypothesis using the Goodness-of-Fit Chi-Square test. First, we will need to state the null and alternative hypotheses for our research question. Since our research question states “Do 11th grade students prefer a certain type of lunch?” our null hypothesis for the Chi-Square test would state that there is no difference between the observed and the expected frequencies. Therefore, our alternative hypothesis would state that there is a significant difference between the observed and expected frequencies. Null Hypothesis (H0 : O) = E(there is no statistically significant difference between observed and expected frequencies) Alternative Hypothesis (Ha : O) 6= E(there is a statistically significant difference between observed and expected frequencies) Using an alpha level of .05, we look under the column for .05 and the row for Degrees of Freedom (remember the Degrees of Freedom = Number of categories −1 = 3). Using the standard Chi-Square distribution table, we see that the critical value for Chi-Square is 7.81. Therefore we would reject the null hypothesis if the Chi-Square statistic is greater than 7.81. Reject(H0 )if X2 > 7.81 4
www.ck12.org
Concept 1. The Goodness-of-Fit Test
Using the table from above, we can calculate the Chi-Square statistic with relative ease.
TABLE 1.3: Frequency Which Student Select Type of School Lunch Type of Lunch Salad Sub Sandwich Daily Special Brought Own Lunch Total (chi-square)
Observed Frequency 21 29 14 36
X2 = ∑
Expected Frequency 25 25 25 25
(O − E)2 /E 0.64 0.64 4.84 4.84 10.96
(0 − E)2 = 0.64 + 0.64 + 4.84 + 4.84 = 10.96 E
Since our Chi-Square statistic of 10.96 is greater than 7.81, we reject the null hypotheses and accept the alternative hypothesis. Therefore we can conclude that there is a significant difference between the types of lunches that 11th grade students prefer. As review, we follow the following steps to formulate and evaluate hypothesis: a. State the null and alternative hypothesis for the research question. b. Select the significance level and use the Chi-Square distribution table to write a rule for rejecting the null hypothesis. c. Calculate the value of the Chi-Square statistic. d. Determine whether to reject or fail to reject the null hypothesis and write a summary statement based on the results.
Lesson Summary 1. We use the Chi-Square test to examine patterns between categorical variables such as gender, political candidates, locations or preferences. 2. There are two types of Chi-Square tests: the Goodness-of-Fit test and the Test for Independence. We use the Goodness-of-Fit test to estimate how closely a sample matches the expected distribution. 3. To test for significance, it helps to make a table detailing the observed and expected frequencies of the data sample. Using the standard Chi-Square distribution table, we are able to create criteria for accepting the null or alternative hypotheses for our research questions. 4. To test the null hypothesis it is necessary to calculate the Chi-Square statistic. To calculate the Chi-Square statistic (x2 ), we use the formula:
X2 = ∑ i
(0i − Ei )2 Ei
where: X 2 = Chi-Square statistical value O = observed frequency value 5
www.ck12.org E = expected frequency value 5.Using the Chi-Square statistic and the level of significance, we are able to determine whether to reject or fail to reject the null hypothesis and write a summary statement based on these results. Supplemental Links Distribution Tables (including the Student’s t-distribution and Chi-Square distribution) http://www.statsoft.com/textbook/stathome.html?sttable.html&1
Review Questions 1. What is the name of the statistical test used analyze the patterns between two categorical variables? a. b. c. d.
the Student’s t-test the ANOVA test the Chi-Square test the z-score
2. There are two types of Chi-Square tests. Which type of Chi-Square test estimates how closely a sample matches an expected distribution? a. the Goodness-of-Fit test b. the Test for Independence 3. Which of the following is considered a categorical variable: a. b. c. d.
income gender height weight
4. If there were 250 observations in a data set and 2 uniformly distributed categories that were being measured, the expected frequency for each category would be: a. b. c. d.
125 500 250 5
5. What is the formula for calculating the Chi-Square statistic? The principal is planning a field trip. She samples a group of 100 students to see if they prefer a sporting event, a play at the local college or a science museum. She records the following results:
TABLE 1.4: Type of Field Trip Sporting Event Play Science Museum
Number Preferring 53 18 29
6. What is the observed frequency value for the Science Museum category? 7. What is the expected frequency value for the Sporting Event category? 8. What would be the null hypothesis for the situation above? a. There is no preference between the types of field trips that students prefer 6
www.ck12.org
Concept 1. The Goodness-of-Fit Test
b. There is a preference between the types of field trips that students prefer 9. What would be the Chi-Square statistic for the research question above? 10. If the estimated Chi-Square level of significance was 5.99, would you reject or fail to reject the null hypothesis?
Review Answers 1. 2. 3. 4. 5. 6. 7. 8. 9.
C A B A 2 X 2 = ∑ (0−E) E 29 33.33 A 20.0 (see table below)
TABLE 1.5: Type of Field Trip Sporting Event Play Science Museum Chi-Square Total
Observed Frequency 53 18 29
Expected Frequency 33.33 33.33 33.33
Chi-Square 12.4 7.0 0.6 20.0
10. Reject the Null Hypothesis
7
