Conditions for performing a significance test about a proportion:
- Random: the data comes from a well-designed random sample or randomized experiment
- 10%: When sampling without replacement, check that n< 1/10N
- Large Counts: Both np0 and n(1-p0) are at least 10
A significance test uses sample data to measure the strength of evidence against H and in favor of Ha. Here are some principles that apply to most tests:
- The test compares a statistic calculated from sample data with the value of the parameter stated by the null hypothesis.
- Values of the statistics far from the null parameter in the direction specified by the alternative hypothesis give strong evidence against H0.
- To assess how far the statistic is from the parameter, standardize the statistic. This standardized value is called the test statistic.
A test statistic measures how far a sample statistic diverges from what we would expect if the null hypothesis H0 were true, in standardized units. That is,
Test statistic = (statistic - parameter)/standard deviation of statistic
Significance Tests: A four step process
State: What hypotheses do you want to test and at what significance level. Define any parameters you use. Plan: Choose the appropriate inference method. Check conditions. Do: If the conditions are met, perform calculations. Compute the test-statistic and find the p-value. Conclude: Make a decision about the hypotheses in the context of the problem
A more comprehensive acronym to use for this test is PHANTOMS. Parameter: State what you are trying to find. Hypotheses: State the null and alternative hypotheses. Assumptions/Conditions: Check to make sure all the conditions are met and which assumptions you can make. Name the Test: Name what type of test you are going to do ([1 or 2]-Sample [t or z]-[interval or test] for population [proportion or mean]) Test Statistic: Calculate the test-statistic. Obtain p-value: Find the p-value using the statistic. Make a decision: Will you reject or fail to reject the null? State conclusion in context: Context is very important!
One sample z test for a proportion
Z = p̂-p0/√((p0(1-p0))/n)
Using a calculator, you can do a z test even though you must show all the above steps as well. Stat -> Tests -> 1-PropZTest p0 = parameter (population value) x = how many of sample n = sample size prop: one or two tailed and which direction
Example (P > 1/12) Prop would be >p0
The power of a test against a specific alternative is the probability that the test will reject H0 at a chosen significance level a when the specified alternative value of the parameter is true.
The power of a test against any alternative is 1 minus the probability of a Type II error (B) that alternative; that is, power = 1 - B
Probability of Type I error: a Probability of Type II error: B
To increase the power of a significance test and detect when H0 is false and Ha is true:
- Increase sample size
- Increase significance level a
- Increase the difference between the null and alternative parameter values that is important to detect. It is easier to detect large differences between the null and alternative parameter values than smaller differences.
There are trade offs to increasing power––it costs more time and money to increase sample size, increasing significance level a increases the chance to have a type I error. In general, you just use your judgement to see which change should happen.