1
Judging Report for CWSF 2001
Final Version – October 3, 2001
Submitted on behalf of the Judging Committee by David Wardlaw, Chief Judge
Preamble
This report attempts to describe all aspects of the judging operations. Material is partitioned into what seemed like logical sections and is presented more or less in chronological order. New features and recommendations are presented in the text in italics and principal recommendations are gathered in a separate section. The Appendix contains two subsections: (a) an analysis of the current distribution of projects among the divisions on the relative & absolute numbers of medals and honourable mentions awarded in 2001; (b) a possible divisional realignment, with concurrent reduction of the number of divisions by one, based on an analysis of the subject matter of the projects in 2001.
The report reveals the magnitude and scope of the judging operation. It seems fitting to begin a report on judging with a tribute to those who made this judging operation possible: all the judges who volunteered their time and expertise and their employers/supervisors who implicitly or explicitly supported this activity in terms of a one- or two-day leave from the workplace. A conservative estimate of the collective commitment by approximately 270 judges on Tues. 15 May and Wed. 16 May is 4000 person-hours which is equivalent to 2 persons working 40-hour weeks for one year.
Table of Contents
Preparation of Chief Judge
The Judging Committee
Recruitment of Judges
Registration of Judges
Timetable for Judging
Judge’s Manual
Hospitality
Project/Exhibitor Statistics
Vetting of Divisional Placement & Category
Project Numbering Scheme
Facilities
Orientation Sessions
Organization of Divisional Judging (Tuesday – Day 1)
Organization of Special Awards Judging (Wednesday – Day 2)
International Projects
Exhibitor Support
Principal Recommendations
Appendix:
a) Effect of Current Divisional Distribution of Projects on Medals/Honourable Mentions
b) A Possible Divisional Realignment
Preparation of Chief Judge
The Chief Judge was recruited in 1996 with no previous experience in organizing the judging operations of a science fair and 2 years as a judge at our regional fair, the Frontenac Lennox & Addington Science Fair, in the mid 1980’s. He became a member of the regional fair judging committee in 1997 and continues to serve in this capacity. He was a judge at the Canada Wide Science Fairs in Regina (1997), Timmins (1998), Edmonton (1999), and London (2000) and served on the Grand Awards team in 1998, 1999, and 2000. Final Reports from North Bay (1996) and Regina (1997) provided valuable insight and useful recommendations.
The Judging Committee
The key to a successful judging operation at a CWSF is a team of dedicated and well-qualified individuals comprising the judging committee. This team was recruited incrementally from 1997 through December 2000. The committee was intentionally “oversized” in anticipation of attrition, which did indeed occur and stabilized at 11 members in by the beginning of February 2001. Recruitment and preparation was done systematically, first by involvement in the regional fair, either as member of the judging committee or as a best-of-fair judge, and then by judging at a CWSF in 1998, 1999, or 2000. All but 2 committee members judged at least one CWSF prior to 2001 and many of those judged 2 CWSF’s. The committee held its first formal meeting before the London fair and had a follow-up meeting immediately thereafter. Monthly meetings commenced in the fall of 2000 and continued until January 2001, at which time biweekly meetings became the norm until mid-April. In the 3 weeks prior to the fair, there were more frequent meetings held as necessary and varying widely in format depending on what type of work needed to be done.
The major responsibilities taken on by the 11 individuals comprising the committee were:
Chief Judge
Special Awards Coordinator & Engineering (EG) Division Chair
Biotechnology (BT) Division Chair
Computing & Mathematical Sciences (CT) Division Chair
Earth & Environmental (ET) Sciences Division Chair
Life Sciences (LV) Division Co-Chairs (shared by two people)
Physical Sciences (PP) Division Chair
Coordinator of Judging in French
Registrar of Judges
Member-at-large (Facilities Coordinator & Safety Officer for Host Committee)
This division of responsibilities allowed the Chief Judge to focus on the management of the overall operation, timelines, communication with the Host Committee, recruitment of judges, contacting delegates to track down missing project information, etc. All committee members also assumed various minor responsibilities in the 4 months leading up to the fair. There was attrition. One committee member moved for employment reasons more than a year before the fair. In the fall of 2000, the would-be EG Division Chair assumed full-time responsibility for facilities and safety. He did however continue to attend committee meetings and his knowledge of the judging operation proved invaluable during planning of the physical set-up, during arrival and set-up of student exhibits, and during the two days of judging. At this point the Special Awards Coordinator took on the additional responsibility of EG Division Chair. This situation (dual major roles) should be avoided because of the high workload associated with organizing Special Awards. In addition, one of the division chairs became, in effect, the Assistant Special Awards Coordinator, during the week before the fair. I recommend that the latter position be established & staffed at the outset. A third committee member, a computer specialist who assumed responsibility for on-line registration, the database, software, etc. for the entire fair in the fall of 2000, resigned when it became clear that managing these electronic aspects was a becoming a full-time job. In lieu of a computer specialist, one of the division chairs took on the task of the computer generation of judging & exhibitor timetables for divisional judging and judges’ on-line registration information was entered manually in a spreadsheet by the Registrar (instead of a database dump). A computer specialist is now essential and future judging committees are advised to have someone who is both qualified for, and dedicated to, this task.
Recruitment of Judges
Recruitment began in the spring of 2000 when letters of invitation to judge at CWSF 2001 were sent to all judges in our regional fair (Frontenac Lennox & Addington) and our two neighbouring sister fairs (Rideau St. Lawrence in Brockville and Quinte in Belleville). Recruitment began in earnest in late January 2001 and early February. A call for judges (in English everywhere; in French to selected institutions) was distributed in electronic (pdf) and/or paper form to the following:
Queen’s University
Royal Military College
St. Lawrence College
Various local & regional industries
Rideau St. Lawrence & Quinte Science Fairs
Several school boards
National Research Council (Ottawa)
Labs in selected federal government departments
A 2nd round of recruiting was undertaken at Queen’s by placement of a story and call for judges in the campus newspaper and by a letter to all science & engineering faculty from the Chief Judge and Vice Principal Academic.
The target for divisional judging on Day 1 was 220. This was based on 215 judges for an estimated 340 projects (5 judgings per project and an average judging load of 8 projects) plus 5 spare judges. Based on a maximum of 40 projects exhibited in French distributed over all categories & divisions, the target was 40-45 judges capable of judging in French. About 1/3 of the projects were in the Life Science division but the number of judges specifying Life Science as their 1st choice was well below 1/3 of the judging pool for Day 1 and the Judging Committee was forced to assign a number of people who had selected Life Science as their 2nd or 3rd choice to this division. It is likely that special effort is required to recruit adequate numbers of Life Science judges. The target for special awards judging on Day 2 was also 220. It was anticipated that a majority would judge on both days but that a significant fraction would judge only one day, and that special award sponsors would provide ~ 30 additional judges for Day 2. These targets were met; statistics are given below in another section.
The judging pool was diverse, consisting of individuals from academia, industry, government labs, and regional school boards. No formal qualifications for judges were imposed, e.g., a certain type of university degree & previous science fair judging experience. After several iterations based on committee discussion and feedback from potential judges, the following statement concerning qualifications emerged; “CWSF judges are strongly encouraged to have previous judging experience (at least one previous local, regional, or national fair) or other experience in evaluating student work (e.g., teacher, professor, teaching assistant). Judges should have a background in science or engineering beyond the high school level (post-secondary degree/diploma/training or suitable equivalent experience). If you wish to judge at CWSF and have no previous experience in evaluating students, you could judge at an upcoming regional science fair in your area.” This advice was offered to potential judges in the calls-for-judges and on the web registration page.
The judges from academia (in our case mostly Queen’s and RMC) consisted of professors, postdoctoral fellows & research associates, a large number of graduate students, and a small number of undergraduate students. Care was taken in assigning undergrads: no more than one per judging team and normally to a team assigned to junior projects. By all accounts the undergraduate students performed well as judges and without them the Judging Committee would have been hard pressed to recruit a sufficient number of judges. Several undergraduates expressed that CWSF judging had been a rewarding and exciting experience. We recommend that senior undergraduate students (completed 3 years of study) be actively recruited as CWSF judges at future fairs.
Registration of Judges
Judges registered on-line at the CWSF 2001 web site in the official language of their choice. Until one week before the regional fair, judges could register for the CWSF or the regional fair or both. Access to the registration page required a password which had been supplied in the call for judges. An email confirmation which echoed the information supplied by the registrant was generated and automatically and copied to the Registrar of Judges and to the Chief Judge. A final confirmation of judges was undertaken in late April and revealed a small but not unexpected amount of attrition. This email informed judges about the upcoming distribution of judging materials and polled them about the judges’ banquet. About 20 people did not respond to this final email confirmation and were telephoned by the Chief Judge. Judges’ registration information was placed in a spreadsheet by the Registrar as it arrived so that the committee could track the number of registrations and distributions of expertise and experience. Incrementally preparing the spreadsheet proved invaluable. A last minute database dump of judges’ information is not advised. Some judges who registered on-line at the first opportunity became anxious when they were not contacted again until late April. This resulted in a small but noticeable number of emails/phone calls to the Registrar from judges wanting to know if they were still needed/registered. In retrospect, it would have been wise to send an intermediate email confirmation to all registered judges in late March.
Timetable for Judging
The detailed timetable is given in the CWSF 2001 Judge’s Manual which accompanies this report. It was created in the fall of 2000 and provided a means of identifying all that would have to be organized, requisitioned, assigned, printed, etc. The following is the short version for judging activities on Monday, Tuesday, and Wednesday during the fair:
MON. 14 MAY:
6-7 pm Registration at PhysEd Centre (pick up gift folder, judging materials, name tag)
7-8 pm Judges' orientation at Dunning Aud.
8-10 pm Project viewing (no students) in gymnasia
TUES. 15 MAY:
8 - 8:15 am Sign-in at Jock Harty Arena
8:15-9 am Team meetings in arena
9am-5pm Divisional judging (lunch provided)
WED. 16 MAY:
8 - 8:15 am Sign-in at Jock Harty Arena
8:15-9 am Team meetings in arena
9am-5pm Special Awards judging (lunch provided)
6:30 pm Judges' Banquet (dinner at 7 pm)
Judge’s Manual
The manual from CWSF 2000 was carefully edited and unclear descriptive passages were rewritten. There were two major changes. First, all forms used by the judging teams for the purposes of arriving at, and submitting, project rankings on Days 1 and 2 were removed; these forms were distributed to team captains as needed during the judging. The only form included was the judge’s mark sheet (both sides). This shortened the manual significantly. Second, the manual was bilingual and printed in standard fashion with the French version starting at one cover and the English at the other cover. This approach reduced printing costs and eliminated the need to maintain separate inventories of the English & French versions and was probably neutral in terms of printing cost versus manuals at previous CWSF’s. It is recommended that the shortened, bilingual version of the Judge’s Manual be used at future fairs.
Hospitality
A hospitality coordinator was brought on board in February 2001, with responsibility for organizing the Judges’ Banquet, communicating with the sponsor of same, communicating with her counterparts on the Host Committee, and arranging beverages, snacks, and meals for the judges from Monday evening through Wednesday afternoon at Queen’s. The hospitality coordinator was not a member of the Judging Committee, nor did she need to be. Removal of this responsibility from the committee allowed it to focus on judging matters. I recommend this division of labour for future judging operations. Judges’ registration and sign-in was handled by a team of volunteers under direction of the registration coordinators from the Host Committee. The hospitality coordinator was explicitly responsible for the following:
- beverages & snack during sign-in and project previewing on Monday evening
- beverages & snacks on continuous basis in breakout area on Tuesday 8:30 am to ~ 9:00 pm and on Wednesday 8:30 am to ~ 4:30 pm (notes: beverage/snack not available in first half of afternoon; snack quantity required for Wed. afternoon is about 2/3 normal amount due to graduated departure of judges throughout the afternoon of special awards judging.)