June 2014
Physics Modeling at the University of Northern Iowa
Grant# XXXXXXXXX, Carver Charitable Trust
Grant amount: $ XXXXXXX
FINAL REPORTto the Carver Charitable Trust Board of Trustees
Overview
This report addresses the University of Northern Iowa implementation of the Iowa Regents Modeling Workshops in physics, which received funding through the Carver Charitable Trust. It is a subsequent grant related to previous workshops receiving funding January of 2011 for the workshop held at Iowa State University and January 2012 for the workshop held at the University of Iowa. This report will communicate the final evaluation of the modeling workshop held at the University of Northern Iowa in the summer of 2013. Comparatively, this workshop was slightly different than the previous two workshops through the inclusion of PRISIMS resources and activities along with the modeling materials.
2012 Profile of UNI Participants
The UNI workshop hosted 22 participants distributed throughout the eastern two thirds of the state. The make-up of participants differed from previous workshops in several ways. First the distribution of teaching experience was much different than the previous workshops at U of I and ISU. The previous workshops were fairly evenly distributed in experience while the UNI workshop had a significant number of participants 12/22 in their first five years of teaching experience. Only three had between five and ten years of experience and the remaining seven had more than 15 years of teaching experience. Additionally, there were fewer in field participants than the previous workshops with only four having significant physics or engineering experience as part of their background. There were a number of participants that did not currently hold a physics teaching position but were preparing to instruct physics at a school with increasing enrollment or who were pursuing endorsement to offer physics at a rural school for the first time. Several participants were encouraged to attend by physics teachers who attended previous workshops with the hope of establishing more continuity in instructional practices and/or for curriculum articulation purposes.
Impact Assessment:
Participant FCI Data
The 2013 UNI participants produced similar trends to the 2011 ISU group and the 2012 U of I group. The gains for the UNI group were figured in similar manner as the previous participant groups and are summarized comparatively as follows. Aggregate UNI participant scores generated an average normalized gain of .43, which is comparable to the mean of the first two workshops. Averages of participants’ pre and post-test averages generate a similar normalized gain of .42. These normalized gains are less than the .55 average of the previous workshop participants figure in a similar manner. Another average was completed excluding participants with initial perfect scores for the reasons outlined in previous reports. When completing this average, the UNI group fairs better with a normalized gain of .65 compared to an average of .54 for the previous groups. This is likely due to the 5 participants well above mastery that finished with perfect scores yielding a normalized gain of 1 thus raising the overall average of this subgroup. Excluding the participants above mastery that finished with perfect scores, produces a result of .55 which is comparable to the average of the previous workshops. As expected, all three workshops have generated participant data consistent with published results from the ASU NSF and MSN Modeling workshops. Tables of all three years of participant FCI scores are presented in appendix 1.
To further evaluate participant data in relation to the grant goal of producing well qualified physics instructors, an evaluation of participants below mastery entering the workshop was conducted. The UNI group started with 10 participants below mastery which is similar to the previous workshops. The major difference between the UNI below mastery group compared to the previous workshops participants was a significantly lower pre-test average. The UNI group started with pre-test average of 14 compared to an average of 18.4 for the previous workshops participants. This is likely due to the greater number of participants who mainly instruct physical science and had less overall experience with physics content. While the UNI group finished with 7 total participants below the mastery threshold, this group began with a significantly lower initial understanding. The group finished with only one participant below the threshold for understanding with most participants in this group finishing just shy of the mastery cut off. Participant scores for this subgroup for all three years follow the FCI scores for all participants in the appendix.
Impact Assessment:
Reflective comments from participants implementation journals
Previous reports for the Iowa Regents Modeling Workshop emphasized weekly reflections during the workshop and mid-year reflections. The UNI participants kept an implementation journal throughout the year that was shared with faculty. The excepts below were cut from participant journals at the conclusion of implementation. In general, similar themes run through each of the three regents modeling workshops. The majority of participants have indicated significant positive change in their professional growth and noticed significant changes in their students’ attitudes, behaviors, and academic abilities. Examples from participant implementation journals follow for those areas.
Reflective comments related to professional growth:
I thought I entered the workshop “knowing” what it was about, but that was not true. I learned so much from the workshop and the professional conversations we have had this year. My thinking and questioning continued to develop as I worked through the physics curriculum this year.
The most significant professional change I have experienced that I can directly relate to my participation in this workshop is my transformation from a teacher centered classroom in physics to a student centered classroom.
Being accepted to the workshop has given me more credibility within my department. It is rather tough to make suggestions for change with a 43 year career teacher. The workshop gave me the experience, deployments, and lab equipment necessary to persuade my colleagues to try something different than they had done before.
I feel my most professional change has been the growth in my approach to teaching as well as becoming more content specific. I pose more questions to students in order to have them become more accountable for their learning. When in the workshop, everything was put back onto me (as a student) to gauge what I was thinking and I had to learn to become an accountable learner. This really opened my eyes t what I was I doing in class and how that needed to change.
The workshop led to a mindshift for me about my approach to teaching and helped me establish a more student-led classroom in which observations, data analysis, critical thinking, and peer review are much more prevalent than they were in my classroom in previous years.
Reflective comments related to student attitudes and behaviors:
Much better understanding as evidenced by FCI. Much better at thinking … like being aware of assumptions and simplifications; like using representation tools; getting away from thinking operation first, and if they do, at least do unit analysis or arithmetical reasoning; like considering the relationships,imagining the physical situation — math less.
I have never felt more refreshed from teaching or as excited about teaching as this year with the modeling curriculum. I am excited to read/learn more to improve my instructional practices. In addition, our district received the TLC grant; I will be able to work with instructional coaches, curriculum coaches, and model teachers to improve upon my instruction.
The students’ perception of science was thoroughly challenged and for some they may not value this rigorous year of science boot camp, but hopefully the thinking and reasoning skills will help them understand how to think for the sake of thinking.
By struggling through some of the labs and having whiteboard discussions, the students were given an opportunity to take collaborative ownership of their work. In past years students showed ownership of work by turning in problem sets, but the whiteboard discussions took the ownership to a new level. Students could be heard defending their work to other groups and engaging in meaningful discussions with each other. Granted it took a lot of battling to get the students to use the whiteboards and truly look at their work, but once the routine was set for our last quarter of classes, the students started to really engage in the discussions. I’m seeing confidence in some of my lower performing students just in the last week that I can directly attribute to the modeling and PRISMS atmosphere.
Implementing what we learned from the workshop last summer was both exciting and overwhelming. The workshop led to a mind shift for me about my approach to teaching and helped me establish a more student-led classroom in which observations, data analysis, critical thinking, and peer review are much more prevalent than they were in my classroom in previous years.
I think the biggest difference for the Physics students was their approach to learning. I got them to buy into the idea of developing problem-solving tools that can be used in a variety of situations. I think the students demonstrated their mastery of this when we got to the projectile motion unit. I simply presented them with a scenario of an object moving in two directions at one time and told them to explain the problem. Each group came up with some ideas, and after a discussion they were able to come to a conclusion as a group, with basically no input from me. After they solved the problem, I pointed out the growth they had demonstrated over the course of the year and made sure they were aware of their problem solving strategies and usefulness of them.
I saw great growth in my physical science student’s ability to think, communicate, and apply their knowledge to different situations from the beginning of the year to the present. Their ability to think like scientists improved greatly! Also, I saw improvement in nearly all of my student’s conceptual exams (Scientific Reasoning and Physical Science Concepts Inventory). Overall, I would say that the PRISMS and Modeling materials, along with the workshop last summer, proved to be very effective!
Impact Assessment:
2013 Participant TPI Results
The Teaching Perspectives Inventory (TPI) was taken prior to the workshop and at the conclusion of the first year of implementation. The delay was intentional to provide an appropriate time frame for actual changes in any perspective to occur. The TPI identifies 5 perspectives in three areas, beliefs, actions, and intentions. For any given perspective the highest score possible is a 45 and the lowest possible is a 9. In general averages for any category are in the mid-thirties and dominant perspectives are those with higher scores. A general descriptor of each perspective follows:
Transmission- Effective teaching requires a substantial commitment to subject matter or content.
Apprenticeship- Effective teaching enculturates students into a set of social norms and habits of mind.
Developmental- Effective teaching is planned and conducted from a learner’s point of view.
Nurturing- Effective teaching assumes long term persistent effort to achieve comes from the heart as well as the mind.
Social Reform- Effective teaching seeks to change society in substantial ways.
The TPI also checks for agreement between a teacher’s beliefs, intentions, and actions within each category but for the purposes of this report, positive changes in these areas are indicative of teachers bringing their beliefs, intentions, and actions into closer alignment.
The UNI group was the first group to produce a near 100 percent return on the pre and post TPI. Previous participants struggled with returning the correct information and or neglected to take the final TPI. When examining the average change in each category we get what we might expect from the content and design of a modeling or interactive engagement workshop. Significant changes in both the apprenticeship and developmental categories occurred for the UNI group. This is expected because the instructional design is based on developing a deep and robust understanding of physics content in relation to what is known psychologically about how people learn. This trend is generally apparent for all three workshops to varying degrees and suggests the designs of the workshops are affecting teachers in areas where you would predict significant effects to occur. When examining UNI participants individual score changes across the apprenticeship and developmental categories, many participants had shifts bringing these two perspectives into a better balance with other perspectives. For a number of the UNI participants, these perspectives went from being a less dominant perspective to the most dominant perspective, with the majority of participants having their most dominant perspective either of these two on their post TPI. Interestingly, the UNI group did show a significant positive change in the nurturing perspective while the other two workshop groups posted a negative and no change in this category. We might speculate this is due to the number of candidates in their first few years of experience but this is not corroborated with individual changes. A number of more experienced teachers brought their nurturing perspectives from being less dominant into balance with other perspectives. A more rigorous comparison of workshop variables and participant make-up would be necessary to address the differences between workshops. The transmission and social reform categories had the least average change amongst UNI participants. For the transmission perspective, positive and negative changes seem to occur as a result of bringing that particular perspective into balance with the other perspectives. The social reform category remained the least dominant perspective for the UNI participants with the positive change being accounted for by several participants with substantial positive changes balanced out by several large negative changes causing a smaller overall positive change. Again, accounting for the differences between each of the workshop groups in areas such as transmission and social reform would require analysis that goes beyond the scope of this report, yet noting the differences seems important in relation to the overall evaluation of the workshops as a whole. A table with all three workshops changes in each category is provided for reference.
A secondary analysis of TPI results consisted of examining the belief, intention, and action changes that occurred in participants. The average change for all three categories was positive and for the majority of participants resulted in a better alignment between all three categories. It can be inferred that the large positive change in beliefs is likely due to the number of participants in their first few years of teaching taking the workshop. This claim is corroborated by significant positive changes in beliefs for participants in their first few years of teaching along with less change for more experienced participants. Perhaps attending the workshop and working with other participants helped affirm their beliefs in relation to any teaching perspective. Changes in intentions and actions cannot be attributed to any subgroup of the participants and seem to be mostly attributed to alignment of beliefs, intentions, and actions as a result of attending the workshop.
In general, the workshop and subsequent attempts at implementation likely affect participants’ perspectives on teaching. The data seem to support a positive effect in both the developmental and apprenticeship perspectives and a better overall alignment of participants’ teaching perspectives and their beliefs, intentions, and actions in relation to effective teaching practices. A table comparing the average changes in teachers TPI scores for all three workshops is provided (Below or in appendix)
TPI Aggregate ComparisonsAverage / Average / Average
Change / Change / Change
TPI category / UNI / ISU / U of I
Transmission / 0.86 / 2.10 / 0.00
Apprenticeship / 3.43 / 1.90 / 1.00
Developmental / 2.43 / 2.50 / 1.50
Nurturing / 2.95 / -1.40 / 0.00
Social Reform / 1.57 / 0.60 / 2.80
Belief Total / 4.24 / 3.00 / NA
Intention Total / 3.05 / 4.14 / NA
Action Total / 3.62 / 2.71 / NA
Impact assessment:
Student FCI Data and VASS Profiling
To be consistent with first year modelers completing a modeling workshop at ASU, a high quality workshop should average a student post-test percentage of 52% or a normalized gain of 0.35. The graphic below is a summary of ASU’s findings for over 7,500 students and serves as the standard for comparing the effectiveness of the Iowa Regents’ Modeling Workshops.
UNI Student FCI Scores:
The UNI participants on average achieved a final post-test average of 50% and an average normalized gain of .32. These results are comparable to results generated by novice modelers attending ASU’s nationally recognized workshops and master’s degree program. However, there were 4 UNI participants producing results that fared worse than traditional instruction with final post-test percentages less than 42% and a normalized gain of less than .21. These participants were from both ends of the participant experience spectrum and were from schools ranging in their diversity and socioeconomic status. Factoring these four individuals out leads to an improvement of the average normalized gain for the remainder of UNI participants to .36 which is slightly greater than what would be expected from published information out of ASU’s Modeling Instruction program. Additionally, 12% of students scored at or above mastery in understanding Newtonian concepts with 32% of students achieving the threshold for having a strong conceptual understanding of Newtonian concepts.