Data Collection Protocol
CCZone Evaluation
September-December 2015
1) Prior to your shift, review this document,the observation quota form, and the observation + interview form. If you have any questions, email Lauren at or call her cell: 303-704-6236.
2) At the beginning of your shift time, pick up your materials from the shelf by Robyn’s desk (in the Education office space).
Make sure you have extra pens/pencils and anything you need for your shift, including a watch or cell phone you can use to check the time.
3) Go to the Cleveland Creates Zone. There should not be a facilitator and/or volunteer working during your observation. If a facilitator is scheduled at the same time as your observation, please make a note on the observation sheet.
4) You will be standing in one of two places during data collection. You will intercept visitors and ask them to participate as they enter the CCZoneat the opening at the top of the stairs. Once a visitor agrees to participate, you will observe from the triangular balcony that juts out above the light sculptures station. Feel free to bringa stoolto the balcony if you need to rest during your shift.
5) Begin with the appropriate papers (observation+ interview forms, the observation quota form, the completed forms envelope, and index cards) onyour clipboard with the observation forms on top, facing up, with interview questions facing downand have a pen/pencil ready.You will need to quickly reference the observation quota form as your group enters the CCZone. Keepthis sheet somewhere you can quickly access; two possible locations are the last paper on your clipboard or underneath the top observation sheet.See what works best for you.
Write your name, the date, and a visitor ID number (begin each shift with #1)on the top of the observation sheet where indicated. If it is the beginning of your shift, do the same on the observation quota form, and mark the time you are beginning your shift.
Please ensure your handwriting is legible!
6) We want a “relatively random” sample of visitors. When you are free (i.e. not observing or interviewing a group), approach the first adultvisitoryou see entering the CCZone. This approach will help make sure we don’t observe and interview the same “type” of people, even on accident (e.g., only those with happy kids, never those who look annoyed, etc.).We are approaching adults(those who appear to you to be 18+) and will be observing and recording responses from youth with adults, but not unaccompanied youth.
7) Approach the adult in the group and say:
“Hi! When you're done exploring this part of the Museum, I would love to talk with you and your group about it for just a few minutes. Would that be ok?"
If they agree, hand them an index card(or another token they can carry while being observed) and continue with:
"Thanks; I’m giving you this card to hold onto while you’re in the space. When you are finished, please bring it back to me. I will come meet you here when you are ready. Feel free to spend as long or as short as you like."
If they do not agree, thank them and let them know you hope they have a great museum visit. Repeat until you find a consenting adult.
8) Once you have consent from an adult, check your clock and note the “Time In” on your observation form, the approximate number of people in the CCZone(i.e., how busy it is), and the number of people in the group you’re observing. Write down the number of adults, number of youth, and list your best guess for the ages of the youth in the group.Check the school group box if the visitors are attending with a school. You will only observe school groups, not interview.
Next, choose one individual in the group to observe. The below box explains how to use the Observation Quota Form to choose the individual to observe.You do not need to inform the groupthat you are only observing one individual.The person you observe can be, but does not need to be the same person who took the card/agreed to participate for the group.
# ofMALES / # of FEMALES
Adults
18 yrs & over / ☐1
☐2
☐3
☐4 / 1
2
☐3
☐4
Youth
Ideally,
7-14 yrs old / 1
☐2
☐3
☐4 / ☐1
☐2
☐3
☐4
Fill out the “Individual Observed” box:check whether they are adult or youth,write their approximate age, and circle their gender.
9) There are columns on the observation form for each of the CCZone components (i.e., rocket launchers, parachutes, light sculptures, etc.) the observed person in the group visited. There is also a column marked “OTHER___”. This space is intended for any new stations which are installed during data collection in the space where the newspaper tube structures are currently located.
First, note the order in which they visited each component (e.g. if they went to the test track first, put 1 in that slot, rockets second, put 2 there, etc.). If a visitor leaves the component, and then comes back, add another number in this slot (e.g. If they went to the test track, then rockets, then test track again, the test track would have 1, 3 and the rockets would be marked 2). If the individual did not visit a component, leave the entire column blank.
At each component, you will tally the number of test trials the observed individual conducted with their creation.For Iteration, only mark one box;check if the visitor returned to the work station at least once to make adjustments or if they did not iterate at that component.If there were other people in the space, indicate whether the project was collaborative, competitive, or both. If the observed individual was alone in the space, leave the Social Interaction segment blank. Use the Project Type segment to indicate whether the individual worked on a solo project, a shared project, or both. If the observed individual did not work on a project, but spent time in the space, leave the Project Type segment blank and explain what they did (e.g. watched a child work, talked on the phone, took pictures, etc.) in the Notes segment.
In the segments labeled Pride Indicators and Challenge Indicators,write anything you see or hear that shows that the individual is proud of their work or encountered difficulty. Use quotation marks to indicate direct quotes.Leave the segment blank if you do not observe anything that fits into that category.
Use theNotes segment to record anything else you see and hear related to the project, making, and iteration (persisting, trying again, and making adjustments when something doesn’t work the first time). Again, use quotation marks to indicate direct quotes and ensure your handwriting is legible! The more information (including detailed notes) that you are able to record on the observation sheets, the better. It can be tricky to watch closely from a distance and record every detail. Remember, this is an evaluation about Making and the Engineering Design Process—prioritize notes about that. It’s ok if you don’t get everything down; just do the best you can.
If a member of the group you are observing asks about the observation, feel free to show/tell them. It is not a secret! Likewise, if anyone wants to learn more about the study, provide them with the study description paragraph and Robyn’s business card.
10) When the observed family/group appears to disengage and begins to leave the CCZone, note the “Exit time” and approximate number of people in the zoneon your observation form.
If the individual was part of a SCHOOL GROUP, this completes the observation. School groups will not be interviewed. Instead, return to the top of the stairs and return to step #5 to begin a new observation.
11) As soon as possible, return to the entry/exit of the space at the top of the stairs. Approach the visitor you gave the index card at the start of their visit and use the opening line on the top of the interview sheet:
“I’d love to talk with you/your group about your experience in the CCZone.
[Then, if appropriate] Do I also have your permission to talk to those under 18 in your group?”
Feel free to let visitors know you expect it to take 5-7 minutes of their time.
***IF A VISITOR DECLINES THE INTERVIEW, DRAW AN “X” THROUGH THE INTERVIEW FORM*** This will let Lauren know that they were observed, but not interviewed. If you know why they refused, it is helpful to note it on the form (e.g. not a native English speaker, on the phone, etc.).
If the visitor refuses, return to step #5 above and repeat.
If the visitor accepts, continue as below.
12) Go through the interview form in the order it is written and ask the questions verbatim.
Please record all answers as clearly and completely as possible and using the visitors’ own words. Use quotation marks to indicate direct quotes).Capture answers from all visitors in a family/group, and star (*) responses from the individual you observed.
If the visitor has questions that you cannot answer, please refer them to a staff member/volunteer who can help, provide them with a copy of the study description paragraph and Robyn’s business card.
13) When you complete the interview, thank them for their time and contributions. You may want to check in to see if there is anything else they’d like to tell you, offer to help them find where they’d like to go next, etc. Being a great ambassador for the museum is never a bad thing!
15) Go back over all your notes to ensure they are complete and legible before you move to your next observation/interview. Take your time with this. Often you think of things you want to add and capture after you’ve had a minute to decompress.
16) Put the completed form in the “Completed Forms” envelope.
When ready, return to step #5 above.Take breaks as needed! This gets tiring!
17) At the end of your shift, please return all materials to the shelf by Robyn’s desk. There will be a space for “completed forms.” Feel free to leave sticky notes for the other data collectors based on what you learned or found helpful during your shift. Also, please log any changes that occur in the space (e.g. new signage, new station, painted walls, etc.) in the notebook. Please be as descriptive as possible.