Food Safety Early Childhood Education Workshop
Meeting Evaluation Report
The second Alabama State University Food Safety Early Childhood Education Workshop took place on March 19 and 20. During the interim between the previous workshop meeting and this one, multidisciplinary teams worked on assigned projects to bring together relevant resources in support of the key elements of the project (content, instructional strategies, "training the trainers" strategies). At this meeting the content area and curriculum expert consultants reported on their activities and accomplishments following the end of the first workshop in February, and then worked on refining and finalizing their recommendations for each of the respective modules.
To evaluate the effectiveness of the second workshop meeting, a post-meeting evaluation form was distributed to the participants. This form represented a modified version of the form utilized in the first workshop, to better reflect the activities and purposes of the second workshop. Where possible, items on the second form were kept parallel to those appearing on the first form, to allow for some comparisons between sessions. Some items appearing on the first evaluation form were dropped from the second form, while other items not appearing on the first form were incorporated into the second form. A total of 21 items were included on the second form, of which 18 were Likert style items and three were open-ended response items.
A total of 9 participants completed and returned usable forms. Responses were transferred from the original forms to an electronic spreadsheet. Using SPSS, analyses were performed on the data captured in the spreadsheet. Analyses consisted of frequency distributions of responses to individual items, summary statistics (means, medians, modes and standard deviations), and comparisons, using the t statistic, of mean responses to those items found on the first and second workshop evaluation forms.
Findings
Examination of the frequency distributions (Table 1) reveals that the participants were generally positive about the project. The first 8 items of the evaluation addressed perceptions of the quality of the workshop, including materials and procedures. For all but one item, the responses were mainly "excellent" or "very good." The lone item for which as many as three respondents selected "satisfactory" addressed the effectiveness of the use of time during the workshop. This may have been a reflection of the structure employed for several steps in the workshop, during which participants worked in small groups. During these small group working sessions, some groups required more time than others, with the likely result that some participants may have felt they were not using their time as productively as they might have liked. The item dealing with clarity of the workshop meeting objectives had an overall positive rating, but also had greater variability of responses, ranging from excellent to unsatisfactory. This response pattern may have been a reflection of some loss of focus toward the end of the first day session.
The second set of 9 items addressed issues of clarity and effectiveness of workshop activities. When asked if they understood and agreed with the goals of the project, the modal response (7) was "agree" with one "strongly agree" and no less favorable responses. When asked if they understand and agreed with the plan for achieving the goals of the project, the responses were a bit less favorable, with fewer "agree" responses, and some "uncertain" responses. There was unanimous agreement with the statements regarding the relevance and utilization of their knowledge and expertise. There was general agreement that the format for creating the "training the other" module was effective, however there was a strong split of responses to the item regarding the effectiveness of the "on-line" forum (four agreeing, four uncertain). There may not have been sufficient time between the inauguration of the on-line forum and the second workshop meeting for the participants to exploit the full potential of the on-line resource. Another possibility is that the participants who were from greater distances may have found the on-line resource more useful, while those participants who were able to meet and communicate with colleagues locally may have considered the on-line forum unnecessary. There was agreement that the collaborative working groups were effective.
The final three items of the evaluation asked participants to judge the degree to which the stated objectives of the workshop had been achieved. There was general agreement that the objective of establishing the content for the food safety training modules had been achieved. There was less agreement regarding the objective to develop a curriculum model for delivering the training modules, with five respondents either agreeing or strongly agreeing, while 3 respondents were uncertain. The third item addressed the objective to develop innovative training materials for implementing the curriculum. Here the respondents were again divided, with five either agreeing or strongly agreeing, and 3 indicating they were uncertain. It should be noted that a much greater portion of the workshop time was devoted to the first of the three objectives, and further work on the second and third objectives remained at the end of the second day of the workshop.
Descriptive statistics were obtained for the responses to each item, principally to identify central tendency characteristics (Table 2). For all but one item (the on-line forum item) the median and modal responses were both 4.0 on a 1 to 5 rating scale (where 5 was most favorable and 1 was least favorable). For the single item with a different central tendency, the median score was 3.5, and the mode was 3. Examination of the standard deviations for the items revealed that while there was general consensus on most items, for a few items there were variations in responses. The previously noted disagreement over the clarity of the workshop meeting objectives was reflected in the largest standard deviation obtained.
Finally, average responses to items were compared between the first workshop in February and the second workshop in March (Table 3). For most items, the average response was very not appreciably different from first session to second session, and the t-test comparisons did not indicate statistically significant differences. For three items, statistically significant differences were found (probability of mean difference occurring by chance less than .05). The first item for which the difference in responses was statistically significant was the first item on the evaluation, which asked for the overall impression of the work-group meeting. The average response was actually slightly lower (4.0) for the second work-group meeting than for the first meeting (4.5). It should be noted that there was a general tendency for respondents to give more favorable responses to most items on the first evaluation form. In the first session, there was a greater emphasis on establishing a sense of community, and communicating the goals of the project, whereas in the second session the majority of the time was devoted to working through difficult and complex problems of content, curriculum, and training methodologies. This shift in emphasis may have prompted the participants to be a bit less "generous" in their responses to the second session. The shift in responses was limited to a small number of participants who moved from "excellent" to "very good", both of which are positive responses.
The second item for which a statistically significant difference in responses was noted was the item concerning interpersonal interactions. Oddly enough, this was one of the items on the second evaluation form to which the responses were most positive. Again, there appears to be a "halo" effect on the first session evaluation, where the most extreme response (excellent) was the choice of nearly every participant. Given the nature of the two sessions, described above, this minor shift in responses does not seem especially problematic.
The final item for which there was a statistically significant difference was the item dealing with the effectiveness of the collaborative teams. Once again, the responses on the second evaluation were on average lower than on the first evaluation. But again, the second session responses were uniformly positive (all 8 agreed with the statement). On the first evaluation, the responses were roughly evenly divided between "agree" and "strongly agree".
Although the differences were not statistically significant, it is worth noting that for all three items regarding the achievement of workshop objectives, the responses trended in the "more positive" direction, as did the item earlier in the instrument regarding the extent to which workshop objectives had been achieved.
Summary
Overall, the perceptions shared by the participants regarding the second work-group meeting seem to support the conclusion that the workshop was successful. The areas in which there were some uncertainties in the minds of some participants were over the effectiveness of the on-line forum, and the extent to which the second (curriculum development) and third (innovative training materials) objectives had been achieved by the end of the session. There was a modest shift from either "excellent" or "strongly agree" responses to "very good" or "agree" responses on three items, from the first session to the second. On the items that specifically addressed the extent to which objectives had been achieved, the shift was in the direction from "uncertain" to "agree".
Table 1
Frequency Distribution of Responses to Evaluation Items
Your overall impression of the March 19-20 work-group meeting:Excellent- 1 / Very good- 7 / Satisfactory- 1 / Unsatisfactory- 0 / Poor- 0
Quality and clarity of information and materials provided about the work-group meeting:
Excellent- 2 / Very good- 5 / Satisfactory- 2 / Unsatisfactory- 0 / Poor- 0
Clarity of the work-group meeting objectives:
Excellent- 2 / Very good- 5 / Satisfactory- 1 / Unsatisfactory- 1 / Poor- 0
Extent to which the objectives of the work-group meeting were achieved:
Excellent- 1 / Very good- 7 / Satisfactory- 1 / Unsatisfactory- 0 / Poor- 0
Extent to which you were satisfied with the interpersonal interactions during the work-group meeting:
Excellent- 3 / Very good- 7 / Satisfactory- 1 / Unsatisfactory- 0 / Poor- 0
Quality of the materials/media used during the work-group meeting:
Excellent- 2 / Very good- 7 / Satisfactory- 0 / Unsatisfactory- 0 / Poor- 0
Extent to which time was used effectively during the work-group meeting:
Excellent- 1 / Very good- 5 / Satisfactory- 3 / Unsatisfactory- 0 / Poor- 0
Overall quality of the work-group meeting:
Excellent- 2 / Very good- 6 / Satisfactory- 1 / Unsatisfactory- 0 / Poor- 0
I understand the goals of the ASU Food Safety Professional Development for Early Childhood Educators project.
Strongly Agree- 1 / Agree- 7 / Uncertain- 0 / Disagree- 0 / Strongly Disagree- 0
I agree with the goals of the ASU Food Safety Professional Development for Early Childhood Educators project.
Strongly Agree- 1 / Agree- 7 / Uncertain- 0 / Disagree- 0 / Strongly Disagree- 0
I understand the plan for achieving the goals of the ASU Food Safety Professional Development for Early Childhood Educators project.
Strongly Agree- 1 / Agree- 5 / Uncertain- 2 / Disagree- 0 / Strongly Disagree- 0
I agree with the plan for achieving the goals of the ASU Food Safety Professional Development for Early Childhood Educators project.
Strongly Agree- 1 / Agree- 4 / Uncertain- 3 / Disagree- 0 / Strongly Disagree- 0
My knowledge and expertise are relevant to the goals of the project.
Strongly Agree- 3 / Agree- 5 / Uncertain- 0 / Disagree- 0 / Strongly Disagree- 0
My knowledge and expertise have been incorporated in the curriculum materials and/or training module designs of the project.
Strongly Agree- 2 / Agree- 6 / Uncertain- 0 / Disagree- 0 / Strongly Disagree- 0
The format of this work-group meeting was effective in facilitating the creation of a “Teaching-the-Other” training module.
Strongly Agree- 0 / Agree- 7 / Uncertain- 1 / Disagree- 0 / Strongly Disagree- 0
The “on-line” discussion group was an effective mechanism for sharing and collaborating between meetings.
Strongly Agree- 0 / Agree- 4 / Uncertain- 4 / Disagree- 0 / Strongly Disagree- 0
The collaborative teams for developing materials and training modules worked effectively.
Strongly Agree- 0 / Agree- 8 / Uncertain- 0 / Disagree- 0 / Strongly Disagree- 0
The work-group meeting achieved its aim of establishing the appropriate content for inclusion in the “teach the other” training modules.
Strongly Agree- 1 / Agree- 7 / Uncertain- 0 / Disagree- 0 / Strongly Disagree- 0
The work-group meeting achieved its aim of developing a curriculum model to be delivered through the “teach the other” training modules.
Strongly Agree- 1 / Agree- 4 / Uncertain- 3 / Disagree- 0 / Strongly Disagree- 0
The work-group meeting achieved its aim of developing innovative training materials to implement the curriculum.
Strongly Agree- 1 / Agree- 4 / Uncertain- 3 / Disagree- 0 / Strongly Disagree- 0
Bottom of Form
Table 2
Descriptive Statistics
Item / N / Mean / Median / Mode / Std. Dev.overall impression of workshop / 9 / 4.0000 / 4.0000 / 4.00 / .50000
quality & clarity of materials / 9 / 4.0000 / 4.0000 / 4.00 / .70711
clarity of objectives / 9 / 3.8889 / 4.0000 / 4.00 / .92796
extent objectives achieved / 9 / 4.0000 / 4.0000 / 4.00 / .50000
satisfaction with interpersonal interactions / 9 / 4.2222 / 4.0000 / 4.00 / .66667
quality of materials & media / 9 / 4.2222 / 4.0000 / 4.00 / .44096
time was used effectively / 9 / 3.7778 / 4.0000 / 4.00 / .66667
overall quality of meeting / 9 / 4.1111 / 4.0000 / 4.00 / .60093
I understand the goals of the project / 8 / 4.1250 / 4.0000 / 4.00 / .35355
I agree with the goals of the project / 8 / 4.1250 / 4.0000 / 4.00 / .35355
I understand the plan for the project / 8 / 3.8750 / 4.0000 / 4.00 / .64087
I agree with the plan for the project / 8 / 3.7500 / 4.0000 / 4.00 / .70711
My knowledge and expertise are relevant to the project / 8 / 4.3750 / 4.0000 / 4.00 / .51755
My knowledge and expertise have been incorporated / 8 / 4.2500 / 4.0000 / 4.00 / .46291
The format of the meeting was effective / 8 / 3.8750 / 4.0000 / 4.00 / .35355
The on-line discussion group was effective / 8 / 3.5000 / 3.5000 / 3.00 / .53452
The collaborative teams worked effectively / 8 / 4.0000 / 4.0000 / 4.00 / .00000
The meeting achieved its aim of establishing content / 8 / 4.1250 / 4.0000 / 4.00 / .35355
The meeting accomplished its aim of developing a curriculum model / 8 / 3.7500 / 4.0000 / 4.00 / .70711
The meeting achieved its aim of developing innovative training materials / 8 / 3.7500 / 4.0000 / 4.00 / .70711
Table 3
Comparison of Mean Responses- Session One and Session Two
Item / session / N / Mean / SD / t / Sign.overall impression / first / 10 / 4.5000 / .52705 / 2.115 / .049
second / 9 / 4.0000 / .50000
quality & clarity of materials / first / 10 / 3.8000 / .78881 / -.579 / .570
second / 9 / 4.0000 / .70711
clarity of objectives / first / 10 / 3.8000 / .78881 / -.226 / .824
second / 9 / 3.8889 / .92796
extent objectives achieved / first / 10 / 3.7000 / .67495 / -1.090 / .291
second / 9 / 4.0000 / .50000
satisfaction with interpersonal / first / 10 / 4.8000 / .42164 / 2.283 / .036
second / 9 / 4.2222 / .66667
quality of materials & media / first / 10 / 4.5000 / .52705 / 1.238 / .233
second / 9 / 4.2222 / .44096
overall quality of meeting / first / 8 / 4.2500 / .46291 / .528 / .605
second / 9 / 4.1111 / .60093
I understand the goals of the project / first / 10 / 4.2000 / .42164 / .402 / .693
second / 8 / 4.1250 / .35355
My knowledge and expertise have been incorporated / first / 10 / 4.6000 / .51640 / 1.495 / .154
second / 8 / 4.2500 / .46291
The format of the meeting was effective / first / 10 / 3.9000 / .73786 / .088 / .931
second / 8 / 3.8750 / .35355
The collaborative teams worked effectively / first / 10 / 4.4000 / .51640 / 2.449 / .037
second / 8 / 4.0000 / .00000
Table 3 (cont'd)
Comparison of Mean Responses- Session One and Session Two
Item / session / N / Mean / SD / t / Sign.The meeting achieved its aim of establishing content / first session / 9 / 3.6667 / .70711 / -1.718 / .111
second session / 8 / 4.1250 / .35355
The meeting accomplished its aim of developing a curriculum model / first session / 9 / 3.4444 / .72648 / -.876 / .395
second session / 8 / 3.7500 / .70711
The meeting achieved its aim of developing innovative training materials / first session / 8 / 3.3750 / .51755 / -1.210 / .246
second session / 8 / 3.7500 / .70711