Amber France
Final Project
Directions: Read the article "Case management educational intervention with public health nurses;
cluster randomized controlled trial," and then answer the following questions. Keep your answers brief. None should be longer than a few sentences. When asked to do a hypothesis test, make sure you define the parameters, state the hypotheses, and state the technical and non-technical conclusions.
1. (4 pts) In a short paragraph, summarize the aim, method, and conclusion of this study.
The purpose of the study was to determine the effectiveness of a collaborative community case management education intervention in terms of reaction, learning, and performance among public health nurses in Taiwan. A cluster randomization controlled trail was conducted with 12 healthcare centers, with 10 being randomized into either the experimental group or the comparison group. The experimental group took a pre-test for baseline data, went through 4 half day educational workshops on community case management, and then took a post-test for data collection. They found that there was a statistically significant difference between the two groups in knowledge, confidence in case management skills, preparedness for case management activities, and self-reported frequency of using the skills attained.
2. (4 pts) Every study begins with a review of previous studies. The authors note several gaps, or limitations, of previous studies. Briefly summarize these gaps and limitations.
Limitations were noted with respect to the evaluation methods used. In the majority of the studies programs were designed and implemented for hospital-based nurses, as opposed to public health nurses, they had small sample sizes, and they had singe group pre- and post-test designs. One study used a quasi-experimental design but did not randomize participants into the groups.
3. (6 pts) The authors describe this study as "a cluster randomized controlled trial with pre- and post-test."
a. What are the clusters? How many clusters were initially available? How many were used for the pilot testing? How many were put in the experimental group? How many were put in the comparison group?
The clusters are the 12 healthcare centers (one in each of the 12 districts of Taiwan). There were initially 12 clusters available and 2 were used for pilot testing and the remaining 10 were then randomized into the experimental group (5 healthcare centers) and the comparison group (5 healthcare centers).
b. Describe how the clusters were randomly assigned to either the experimental group or the comparison group.
They were randomly assigned to either the experimental group or comparison group by card allocation. Each health centers name was on a card and a third party allocated the cards to one of the two groups.
c. Briefly describe what is meant by pre- and post-test in this study.
The participants took a pre- and post-test that consisted of questions pertaining to case management knowledge, case management skills, case manger role activities, and demographic data and evaluation data. The case management knowledge section was multiple choice, the case management skills and case manger role activities sections was answer by using rated scales from 1=very low to 5=very high. The evaluation portion of the demographic data and evaluation data section included a rated scale of 1=not at all to 5=very often.”
d. How many total participants were in the experimental group? How many were in the control group?
There were 85 total participants in the experimental group and 76 in the comparison group.
4. (2 pts) In chapter 2 of the textbook, we defined an experimental study as one in which something is done to a group. Briefly describe what the "something done" to the experimental group is in this study.
The intervention that is done to the experimental group in this study is the 4 half day educational workshops on community case management.
5. (6 pts) On page 2238, the authors discuss validity and reliability. Read the short paper "Reliability and Validity" by Michael J. Miller.
a. Describe in your own words what are meant by reliability and validity.
Reliability is getting a consistent score each time it is tested or measured. Validity measures what it is intended to measure. For example a scale measures you weight the same each time you go on it and it is measuring your actual weight. This would be valid because it is reliable and measuring what it is intended to measure.
b. Describe why each idea is important in a measurement instrument.
It is important to have reliability because you are getting a consistent measurement but it is also important to have validity because you are getting a measurement of what you are intending to measure. For example a scale to measure weight may be reliable because it gives a consistent measurement each time but it may not be valid because the measurement is off by 5 pounds each time. You need to have both reliability and validity in a measurement instrument to get accurate results.
6. (6 pts) On page 2241, the authors discuss study limitations and bias. Read the definitions of different types of bias given at the Statistical Analysis Handbook.
a. In your own words, summarize the five types of bias defined.
The five types of bias are selection bias, recall bias, estimation bias, systemic bias, and observer bias.
Selection bias is when you select a sample that is not representative of the population of interest.
Recall bias can occur when subjects are asked to recall information after an event. It is typically less reliable. An example of when recall bias can occur is when filling out questionnaires.
Estimation bias is the difference between the population value of estimated from a sample value.
Systemic bias is bias that is generated during the system of collecting the information. For example if you are using equipment it may not be calibrated correctly.
Observer bias is when the observer creates some bias, which is typically unintentional. For example behaviors may be altered in a study with the presence of an observer.
b. Which of these types of bias do the authors of the study identify as possibly being present?
The authors indicate that observer bias could be present since neither the participants nor the researchers were blinded to the study and the PHN may have changed their responses to make a good impression. They also indicated that recall bias could be present because they are filling out self-reported questionnaires which can lead to overestimation or underestimation of results. There also could have been some systematic bias because the researchers did not include a level 4 evaluation or assess the follow-up effect after the educational intervention.
7. (6 pts) Table 2 on page 2240 is titled "Descriptive statistics and pre-test differences in outcome variables."
a. Briefly describe what the purpose of this table is. Specifically, what is the purpose of the hypothesis tests?
The purpose of the table is to show the results that indicate if there is a statistically significant difference between the intervention group and the comparison group in regards to the five outcome variables in the baseline assessment.
b. Based on the given results, is there any significant difference between the experimental group and the comparison group in terms of mean pre-test scores? Briefly describe how you came to this conclusion.
There was not a statistically significant difference between the experimental group and the comparison group in terms of the pre-tests scores because all of the p-values are much greater than 0.05.
To analyze the data, the authors use three main tools: t-test, 2 c -test, and ANOVA. They also use a Mann-Whitney U-test. Specifically the t-test used is a 2-sample t-test, and the 2 c -test is the 2 c -test of independence. The ANOVA test used in this study is a two-way ANOVA test, which is an idea we did not discuss in this class.
Table 1 on page 2240 gives the results from 5 different hypothesis tests for comparing the experimental group to the comparison group. We will replicate these calculations using the Excel Chapter 7 workbook.
Answer the questions 8 - 13 below in an Excel workbook and submit it with your answers to the
questions above.
8. (4 pts) The first test in Table 1 under the heading "Educational level" is used to compare the educational levels of members of the two groups. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value. Based on this result, is there a significant difference between the education levels of the two groups?
State the Hypotheses
H0: There is no difference in education level between the intervention group and the comparison group.
H1: There is a difference in education level between the intervention group and the comparison group.
State the Conclusion
Technical: Do not reject H0
Non-technical: There is not a statistically significant difference in education level between the two groups.
There does not appear to be a significant difference in education level between the two groups.
9. (4 pts) The second test in Table 1 under the heading "Previous training" is used to compare the amount of previous training of the two groups. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value. Based on this result, is there a significant difference between the amounts of previous training of the two groups?
State the Hypotheses
H0: There is no difference in amounts previous training between the intervention group and the comparison group.
H1: There is a difference in amounts previous training between the intervention group and the comparison group.
State the Conclusion
Technical: Do not reject H0
Non-technical: There is not a statistically significant difference in amounts previous training between the two groups.
There does not appear to be a significant difference between the amounts of previous training of the two groups.
10. (4 pts) The third test in Table 1 under the heading "Age (years)" is used to compare the mean age of one group to the other. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value (note that your values may be slightly different that those given in the article due to rounding errors). Based on this result, is there a significant difference between the mean age of the two groups?
Define the parameters
µ1 = mean age of all subjects in the intervention group
µ2 = mean age of all subjects in the comparison group
State the Hypotheses
H0: µ1 = µ2
H1: µ1 ≠ µ2
State the Conclusion
Technical: Do not reject H0
Non-technical: There is not a statistically significant difference between the mean age of the intervention group and the comparison group.
There does not appear to be a significant difference between the mean ages of the two groups.
11. (4 pts) The fourth test in Table 1 under the heading "Years in nursing" is used to compare the mean years in nursing of one group to the other. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value (note that your values may be slightly different that those given in the article due to rounding errors). Based on this result, is there a significant difference between the mean years in nursing of the two groups?
Define the parameters
µ1 = mean years in nursing of all subjects in the intervention group
µ2 = mean years in nursing of all subjects in the comparison group
State the Hypotheses
H0: µ1 = µ2
H1: µ1 ≠ µ2
State the Conclusion
Technical: Do not reject H0
Non-technical: There is not a statistically significant difference between the mean years in nursing of the intervention group and the comparison group.
There does not appear to be a significant difference between the mean years in nursing of the two groups.
12. (6 pts) The fifth test in Table 1 under the heading "Years in public health nursing" is used to compare the years in public health nursing one group to the other. The given test statistic and P-value are from a Mann-Whitney U-test. (Note that not enough data are given for us to replicate these calculations.)
c. Based on the given P-value, is there a significant difference in years in public health nursing between the two groups.
Based on the P-value of 0.530 there is not a significant difference in years in public health nursing between the two groups as the P-value is much greater than 0.05.
d. The Mann-Whitney U-test is similar to the 2-sample t-test. Use the appropriate worksheet in the workbook Chapter 7 to use a 2-sample t-test on this data. Does this give a different result that the Mann-Whitney U-test? Speculate as to why the authors chose to use the U-test rather than the t-test.
Define the parameters
µ1 = mean years in public health nursing of all subjects in the intervention group
µ2 = mean years in public health nursing of all subjects in the comparison group
State the Hypotheses
H0: µ1 = µ2
H1: µ1 ≠ µ2
State the Conclusion
Technical: Do not reject H0
Non-technical: There is not a statistically significant difference between the mean years in public health nursing of the intervention group and the comparison group.
The 2-sample t-test does give a different result than the Mann-Whitney U-test. The authors would have chosen to use a U-test rather than a t-test because the samples are form the same population and likely have the same population shape. The samples also would not have a normal distribution resulting in using a nonparametric test.
13. (4 pts) On page 2239, under the subheading "Effect on knowledge," the authors compare the total knowledge mean score for the intervention group (also called the experimental group) to the comparison group. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value (note that your values may be slightly different that those given in the article due to rounding errors). Based on this result, is there a significant difference between the mean scores of the intervention group and the comparison group?