ED635 Action Research

ED635 Action Research

Data Collection: Observation

Data Collection Observation Project

The process for the observation project:

a.  Read the article Seeing what you normally don’t see by Zhang, Lundeberg and Eberhardt

that is posted as a pdf.

b.  Plan a small study that will use observation instruments (and strategies) to collect data

A planning template is provided. Complete the template. Once you have the problem statement and question done, the rest should fall into place and align.

c.  Design original or modify existing observation instruments (and methods) to collect data. If you use an existing (published) instrument make sure you identify the source. See the posted Power Pt on Observation and Ch 3 in the textbook to help make choices.

d.  Pilot the use of any instruments- if possible- time permitting.

For this project, the pilot is the best way for you to get used to observing and collecting data using the instruments you have selected. The pilot does not need to be done in the same setting the study will be done. But it should be similar enough to give you a good idea of how your instruments will work. This is optional given our time constraints. In an actual research study, the pilot will be essential to the success, validity and reliability of data, of the research. The pilot helps discover flaws.

e.  Conduct the observation study

Conduct the study using the instruments/tools and strategies you prepared. You can switch instruments if you find they do not work at all. But if they are reasonably good, stick with them. If they do not work, that will come up in the analysis section. It is okay if it turns out that way. Research is kind of messy sometimes!

It is wise to -

Keep a journal or log of what happens and

Try to write in the journal the same day you collect data.

f.  Analyze the data and instruments

g.  Reflect: Think about different aspects of the study.

Observation Study Plan

Step 1: Create a plan

Observation Data Collection Plan Template
Problem Statement: Describe the problem that you want to investigate.
Purpose Statement: Write the purpose of the study. Purpose statement generally begin with “The purpose of the study is …………..”.
Research question: Based on the problem you described and the purpose statement, write a research question that will allow you to use observation to help research it.
Setting: Provide a detailed description of where and when the study will be conducted. If this is a group study completed in several different settings- each person describes their own setting.
The Sample (if this is a group project done in different settings, each member writes their sample information)
·  Description: Who will your subjects be? Give the basic demographic information
·  How selected: How will your subjects be selected to participate in the study?
Data Collection
Method: Observation / Instruments/tools
List the three different observation tools and/or strategies to collect data (ancedotal notes, checklist with tally count, video...)
Why did you select them?
Are these original or existing (published) instruments?
How you will proceed to use them?
Data Analysis strategies:
Select at least two analysis strategies that are a good fit for this study.
Resources to use to help you select:
Look at course Data Analysis Basics guidelines Chapter 4 in the text. Also- try
http://gse.gmu.edu/research/tr/tr_process/tr_analysis
Bias considerations: What potential biases do you need to consider for this study and these observation instruments?
Ethics: Explain any ethical issues related to the study.

Step 2: Conduct the observations: (Pilot if you have time)

Step 3: Data analysis

Step 4: Reflection: what was learned about observation as a process for collecting data.

Observation Data Collection Project Criteria (55 points)

5 / 3 / 1 / 0
Problem x 2 / Detailed description, rich portrayal of problem / General description, of the problem / Vague description of the problem. / No problem described.
Purpose x 2 / Contains well articulated purpose / Purpose is stated / Purpose is vague / Purpose is poorly worded, not easily understood
Context/Setting / Detailed description, rich and accurate portrayal of setting where data collection takes place / Some description and fairly accurate portrayal of relevant of setting where data collection takes place / Little description and/or inaccurate portrayal of relevant of setting where data collection takes place / No evidence of understanding of what relevant of setting where data collection takes place.
Question / Question is clear, researchable,
and is accurately defined. / Question is researchable, adequately defined. / Question is vague, not fully researchable / Question is poorly defined or undefined;
Not researchable
Sample / Sample and selection is clearly described, all relevant demographics included / Sample and selection is described, the most relevant demographics included / Sample and selection is vaguely described, demographics included / Sample and selection is not described or is confusing, demographics incomplete or missing
Data Tools- which ones
X2 / Evidence of three or more forms of observational data collection tools or strategies with clear justification of why those tools are used
(how they will help answer the question) / Evidence of three forms of observational data collection tools or strategies with simplistic justification of why those tools are used / Evidence of less than three or more forms of observational data collection tools or strategies with minimal or poor justification of why those tools are used / Evidence of one form of observational data collection tools or strategies with no justification of why it is used
Bias / The most likely forms of bias are identified / Likely forms of bias are identified / Forms of bias are identified but an important source of bias is omitted / No forms of bias are identified OR biases identified are not a match for the study
Ethics / Most important ethics factors are identified / Ethics factors are identified / Ethics factors are identified but an important one is missed / No ethics factors identified

Analysis of Observation Data and Tools (30 pts)

Because we are learning about research methods, research instruments as well as collecting actual data there are two analyses to undertake. One is analysis of the data collected. The other is analysis of the tools and strategies used to collect the data.

1. For data analysis, use the guidelines and information from the resources provided in the plan and in the data analysis general guidelines. I will specifically be looking for the following:

·  One page data analysis (being concise) plus additional for graphs, table, charts or other representations.

·  Use evidence from data collection. You have three sources of evidence ( like anecdotal records, checklists with tallies, )for one question.

·  Interpretations of data- considering you are doing a limited study do not make major claims!

·  Well written.

(21) pts

2. For analysis of instruments or tools and strategies, use the guidelines below.

Analysis of alignment between question, purpose, observation data collection, tools (and methods) (9 pts).

For each observation tool and method used, determine its strengths and weaknesses. Based on that, give each one a score in relation to how good a fit the tool turned out to be, given the question. Justify the score.

Scoring:

5 = Best fit, worked exactly like we thought it should, data from this tool(method) will be valid and reliable

4 = Very Good fit, with just a minor tweaking it would have been ideal

3 = Good, worked out okay, instrument needed a bit of help or not aligned as well as I thought

2 = Fair, not really a good fit for this study

1 = Poor, I don’t know why we used this, it does not work at all given the question, data collected using this is not meaningful

See the example below

Observation Instrument (and method) / Strengths / Weaknesses / Score and Justification
Checklist ( with tally marks) / Yes or No- only two choices
Easier to use when observing the entire class / Made too many items on the checklist
A few items did not really align well with the question / 3-4 This would have been a best fit except that there were too many items on the checklist to keep track of. Two items did not align well. I may have missed some info because of that.

Reflection on Observation as a Data Collection Method(15 points)

This is where you put it all together.

Select two or three ideas to focus on for the reflection.

Consider the following as possible prompts.

·  What did you learn about using observation as a research technique?

·  What went well? What did not go as well as you hoped?

·  What might deserve careful consideration when using observation?

·  How easy or difficult is it to do a study based on observation?

Suggestion: Read (It is a quick read)

Seeing what you normally don’t see by Zhang, Lundeberg and Eberhardt

It is posted as an optional reading.

Guidelines:

I am specifically looking for a

·  meaningful reflection,

·  with careful thought,

·  that makes connections, and

·  uses specific examples from the study to support ideas.

No more than two pages.