Final Project: Andrea Eden Shingleton 2/26/15

1. (4 pts) In a short paragraph, summarize the aim, method, and conclusion of this study.

In order to ascertain the usefulness a joint education intervention between of clinical and academic nurses in need of case management skills, researchers in this study designed a cluster randomized controlled trial in Taiwan. The 10 districts were randomly split into five control and five intervention groups where all participants were evaluated with a pre and post test. The investigators found that the collaboration between academic and clinical nurses is an effective strategy to prepare nurses for quickly shifting roles as case managers.

2. (4 pts) Every study begins with a review of previous studies. The authors note several gaps, or limitations, of previous studies. Briefly summarize these gaps and limitations.

After identifying nineteen studies that evaluated the case management continuing professional education program (CMCPE), the author recognized several limitations. First, all of the studies were hospital-based. Next, the majority of the studies included small convenience samples, none of which were in Taiwan. All of the studies with pre and post test design were single groups and not randomized, limiting the conclusions due to lack of a control group.

3. (6 pts) The authors describe this study as "a cluster randomized controlled trial with pre- and post-test."

a. What are the clusters? How many clusters were initially available? How many were used for the pilot testing? How many were put in the experimental group? How many were put in the comparison group?

The participants were grouped based on geographical location/hospitals or clusters. Of the twelve clusters available, two were used for the pilot study. The five of the remaining ten administration districts were indiscriminately chosen to receive the intervention and the other five were designated as the control groups.

b. Describe how the clusters were randomly assigned to either the experimental group or the comparison group.

The random distribution method incorporated in this study used a third party to draw cards with each hospitals name on it and randomly allocate them to the experimental group and to the comparison group.

c. Briefly describe what is meant by pre- and post-test in this study.

The pre test was administered at the beginning of the study and prior to the intervention to establish a baseline. The post test was given at the end of the study and after the collaborative educational intervention was completed to determine the effectiveness. The tool used was a Chinese-language questionnaire used to access case management knowledge, skills, role activities and demographic and evaluation data.

d. How many total participants were in the experimental group? How many were in the control group?

There are 85 total participants in the experimental group. There are 76 total participants in the comparison group.

4. (2 pts) In chapter 2 of the textbook, we defined an experimental study as one in which something is done to a group. Briefly describe what the "something done" to the experimental group is in this study.

The academic nurses conducted four half-day sessions every two weeks in the five experimental districts in case management continuing professional education program (CMCPE) as the intervention for the experimental group.

5. (6 pts) On page 2238, the authors discuss validity and reliability. Read the short paper "Reliability and Validity" by Michael J. Miller.

a. Describe in your own words what are meant by reliability and validity.

Reliability of a research tool like a questionnaire can be defined as the dependability of the tool to do what it was designed to do. A reliable instrument will produce the same results or scores consistently. Validity of an instrument can be defined as the bringing about the intended measure of interest. A valid tool will accurately assess the desired characteristic.

b. Describe why each idea is important in a measurement instrument.

The reliability of a tool used in research is important so that the researcher can depend on it to give consist data. The validity of the tool is important, so that the results are a true reflection of the desired trait. Together, reliability and validity assure the investigator of sound conclusions, legitimate recommendations and reproducible results.

6. (6 pts) On page 2241, the authors discuss study limitations and bias. Read the definitions of different types of bias given at the Statistical Analysis Handbook.

a. In your own words, summarize the five types of bias defined.

There are five types of bias; selection, recall, estimation, systematic and observer. Bias or prejudice can be deliberate or unintentional in research studies of all types. When selecting participants it is important to find a sample representative of your population and not just easy to collect data and willing participant to avoid selection bias. Also, meta-analysis studies are susceptible to publication bias meaning the only available studies to analyze are in print and the investigator’s preconception of the outcome may guide him to only select one’s that confirm a particular viewpoint. When researchers are using questionnaires they must be on alert for recall bias in their results because respondents often over-estimate or under-estimate or omit important data. It seems that case-control studies are especially predisposed to bias due to the need for participants to recall information after the event. Bias of the estimator happens when there is a difference between the true population value and the estimated sample value. Systematic or usage bias occurs when specialized equipment is not calibrated effectively or is faulty due to unknown environmental factors. Observer bias is created when witnessed data is victim to the subjective nature of the observer.

b. Which of these types of bias do the authors of the study identify as possibly being present?

The researchers and the participants were not blinded to the intervention introducing potential selection and observer bias. The authors used a questionnaire potentially introducing recall bias. In the article, the researcher affirms that social desirability bias cannot be excluded if respondents changed their answers to make a good impression on the academic researcher.

7. (6 pts) Table 2 on page 2240 is titled "Descriptive statistics and pre-test differences in outcome variables."

a. Briefly describe what the purpose of this table is. Specifically, what is the purpose of the hypothesis tests?

Table 2 shows the difference characteristics and their statistical significance between the experimental and control groups, which were identified from the pre- test. These characteristics included; knowledge (the result of part one of the questionnaire,) skill confidence and frequency (the result of part two of the questionnaire,) activity participation and frequency (the result of part three of the questionnaire.)

b. Based on the given results, is there any significant difference between the experimental group and the comparison group in terms of mean pre-test scores? Briefly describe how you came to this conclusion.

There are no statistically significant differences from the pre test between the experimental and the control group. When looking are the p values in each category the results are all greater than 0.05, denoting no statistical significance.

To analyze the data, the authors use three main tools: t-test, 2  -test, and ANOVA. They also use a Mann-Whitney U-test. Specifically the t-test used is a 2-sample t-test, and the 2  -test is the 2  -test of independence. The ANOVA test used in this study is a two-way ANOVA test, which is an idea we did not discuss in this class.

Table 1 on page 2240 gives the results from 5 different hypothesis tests for comparing the experimental group to the comparison group. We will replicate these calculations using the Excel Chapter 7 workbook.

Answer the questions 8 - 13 below in an Excel workbook and submit it with your answers to the questions above.

8. (4 pts) The first test in Table 1 under the heading "Educational level" is used to compare the educational levels of members of the two groups. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value. Based on this result, is there a significant difference between the education levels of the two groups?

Define the parameter(s)

Difference of education levels between the experimental and comparison group

State the Hypotheses

H0: the Educational levels are independent of group designation

H1: the Educational levels are dependent of group designation

State the Conclusion

Technical: Reject H0 if 2 > 9.49 (Table 3, p. 279 x2 critical value 9.49)

2 =3.07<9.49 do not reject the H0

Non-technical: The difference in educational levels are independent of group designation.

Based on this result, there is not a significant difference between the education levels of the two groups.

9. (4 pts) The second test in Table 1 under the heading "Previous training" is used to compare the amount of previous training of the two groups. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value. Based on this result, is there a significant difference between the amount of previous training of the two groups?

Define the parameter(s)

Difference in previous training between the experimental and comparison group

State the Hypotheses

H0: previous training is independent of group designation

H1: previous training is dependent of group designation

State the Conclusion

Technical: Reject H0 if 2 > 3.84 (Table 3, p. 279 x2 critical value 3.84)

2 =0.10 <3.84 do not reject the H0

Non-technical: Previous training is independent of group designation.

Based on this result, there is not a significant difference between the amount of previous training between the two group.

10. (4 pts) The third test in Table 1 under the heading "Age (years)" is used to compare the mean age of one group to the other. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value (note that your values may be slightly different that those given in the article due to rounding errors). Based on this result, is there a significant difference between the mean age of the two groups?

Define the parameter(s)

µ1 mean age of experimental group

µ2 mean age of comparison group

State the Hypotheses

H0: µ1 = µ2

H1: µ1  µ2

State the Conclusion

Technical: reject H0 if z -1.960 or if z  1.960 (table1c, p.275)

Do not reject H0 0.46 <1.960

Non-technical: We do not have statistically significant evidence to confirm that there is a difference in mean ages of the two groups.

Answer to follow-up question:

Based on the results, there is not a significant difference between the mean age of the two groups.

11. (4 pts) The fourth test in Table 1 under the heading "Years in nursing" is used to compare the mean years in nursing of one group to the other. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value (note that your values may be slightly different that those given in the article due to rounding errors). Based on this result, is there a significant difference between the mean years in nursing of the two groups?

Define the parameter(s)

µ1 mean number of years in nursing in experimental group

µ2 mean number of years in nursing in comparison group

State the Hypotheses

H0: µ1 = µ2

H1: µ1  µ2

State the Conclusion

Technical: reject H0 if z -1.960 or if z  1.960 (table 1c, p.275)

Do not reject H0 0.94 <1.960

Non-technical: We do not have statistically significant evidence to confirm that there is a difference in mean years in nursing of the two groups.

Based on the results, there is not a significant difference between the mean years of nursing of the two groups.

12. (6 pts) The fifth test in Table 1 under the heading "Years in public health nursing" is used to compare the years in public health nursing one group to the other. The given test statistic and P value are from a Mann-Whitney U-test. (Note that not enough data are given for us to replicate these calculations.)

c. Based on the given P-value, is there a significant difference in years in public health nursing between the two groups.

Based on the given P-value, there is not a significant difference in years in public health nursing between the two groups.

d. The Mann-Whitney U-test is similar to the 2-sample t-test. Use the appropriate worksheet in the workbook Chapter 7 to use a 2-sample t-test on this data. Does this give a different result that the Mann-Whitney U-test?

No, both the U-test and the t- test result do not have statistically significant evidence to confirm that there is a difference in mean years in public health nursing of the two groups.

Speculate as to why the authors chose to use the U-test rather than the t-test.

I believe the author choose the Mann-Whitney U-test because unlike the t-test, it does not assume normal distribution and it is useful for two samples with different numbers of observations.

13. (4 pts) On page 2239, under the subheading "Effect on knowledge," the authors compare the total knowledge mean score for the intervention group (also called the experimental group) to the comparison group. Use the appropriate worksheet in the Chapter 7 workbook to calculate the test statistic and P-value (note that your values may be slightly different that those given in the article due to rounding errors). Based on this result, is there a significant difference between the mean scores of the intervention group and the comparison group?

Define the parameter(s)

µ1 mean knowledge score of experimental group

µ2 mean knowledge score of comparison group

State the Hypotheses

H0: µ1 = µ2

H1: µ1  µ2

State the Conclusion

Technical: reject H0 if z -1.960 or if z  1.960 (table1c, p.275)

Reject H0 15.27 > 1.960

Non-technical: We have statistically significant evidence to confirm that there is a difference mean knowledge scores of the two groups.

Based on the results, there is a significant difference between the mean knowledge scores o f the two groups.