Data Nuggets
OBSERVATION PROTOCOL CODING MANUAL
Draft 5 - Edited 3/6/17
We describe what to look for in the classroom as indicative of the 8 SEPs from the NGSS. It is important to familiarize yourself with the full definitions and examples of the codes prior to observing the lesson. The codes are described in detail below; only the checkboxes appear on the actual record sheet.
Code Definitions
Code / Description*SEP1: Asking questions / Students ask and refine science questions and/or develop hypotheses based on observations.
Notes:
● We define a scientific question as distinct from other types of questions in that the answer lies in explanations supported by empirical evidence, including evidence gathered by others through investigation (NGSS Appendix F). Scientific questions can be answered by data-supported claims. Leads into the claim-evidence-reasoning (CER) framework in SEP6.
● NGSS (Appendix B, p. 52): “Asking question and defining problems also involves asking questions about data...It is important to realize that asking a question also leads to involvement in another practice. A student can ask a question about data that will lead to further analysis and interpretation.” But for our purposes of coding we are not going to code this way because we could be coding this all the time otherwise. We will use the more narrow definition of question as “scientific question” that leads to significant additional analysis and interpretation.
● We define hypothesis as a proposed explanation for an observation, which can then be tested with experimentation or other types of studies. Hypotheses can be broadened beyond the scope of the experiment and tied to a larger mechanism of which the experiment is only testing one part. In the classroom, hypotheses are often confused with predictions, or what the student thinks he/she will observe in an experiment. Teachers can prompt students to think about their hypothesis by asking them “why” (mechanism) they think they will see their particular predictions (Strode 2015).
● SEP1 vs. SEP3: Sometimes teachers and/or students use the word “hypothesis” when they really mean “prediction.” For example, the teacher may ask students to write their hypotheses in their notebooks before they begin an investigation, but the students actually write predictions about what will happen in the investigation. In this case, code the activity based on what students are actually doing and not the words they use; i.e., code an instance like that described here in SEP3, Planning and Carrying Out Investigations (which specifies “making predictions”).
● SEP1 vs. SEP6: Students may be revising and developing scientific questions and hypotheses while simultaneously working on other practices. For example, they may be constructing an explanation (SEP6) that inspires them to think more broadly about their hypothesis. Additionally, data interpretation and analysis may cause them to assess the validity of their hypothesis. The timing is important. If it is after the investigation then it is SEP6, but if it is before the investigation it is SEP1.
Observing any of the following would be coded for SEP1. Students working collaboratively or individually:
⬜ develop scientific questions.
⬜ develop hypotheses.
⬜ evaluate the strength of scientific questions. This includes general discussions of the merit of a particular question. For example, discussions could focus on what makes a strong scientific question, such as it cannot be answered with a simple yes or no, has a scientific component, and is testable and relevant.
⬜ evaluate strength of hypotheses. This includes general discussions of what makes a strong hypothesis, for example that it provides an explanation for a particular phenomenon and leads to testable predictions.
SEP2: Developing & Using Models / Students are engaged in using a model as a tool to make sense of a phenomenon. A model could be a three-dimensional representation, computer simulations, a diagram, physical representation, or a mathematical or computational formula (mathematical representations).
Notes:
● Graphs are typically NOT used as models; reasoning with graphs will likely be coded in SEP 4, Analyzing and Interpreting Data. [This disagrees with the work of Mayes, but we will follow the definition of models as stated by NGSS.]
● SEP 2 vs. SEP5: A mathematical formula would count for SEP2 if the students were considering how the different parts of the function related to the biological context. It would be coded as SEP5 if they are simply carrying out the functions of the model (formula) and performing mathematical calculations.
● Be particularly cautious when coding lab exercises or teacher demonstrations. These often involve instances of the actual phenomenon, not a model. For example, using a model organism is NOT a model as described by the Framework and NGSS.
● If the teacher simply draws something on the board or shares it with students it would not count. It counts if the teacher prompts them to engage with the model (talk about it or reference it in some way), or if the student points to specific parts of the model to help make their case.
Observing any of the following would be coded for SEP2. Students working collaboratively or individually develop and use model(s) to:
⬜ generate data. This includes students using simulations, such as a climate model, to collect data.
⬜ make predictions. If they call it a hypothesis but it is in fact a prediction, code it here.
⬜ explore and/or explain phenomena. If students use a model to construct their explanations, code both SEP2 and SEP6.
⬜ evaluate strengths of model. This includes students comparing the merits and limitations of two different models of the same system. In this case (comparing two different models), SEP 7 will be coded as well as SEP2.
SEP3: Planning & Carrying Out Investigations / Students are engaged in planning and conducting an investigation that will help them answer their science questions by producing data that will serve as evidence. Students should design investigations that generate data to provide evidence to support claims they make about phenomena (NGSS Appendix F). Students design ways to investigate research questions, including choosing appropriate variables, techniques, and tools to gather, record, and analyze data (ISIOP). Students make predictions of what they expect to observe in their experiment. Refer to description of prediction vs. hypothesis in SEP1.
Notes:
● Simply following the steps provided by the teacher or textbook is not enough to code SEP 3. Students must generate their own ideas for an investigation and demonstrate they understand what elements make up a manipulative experiment or observational study.
● In order to code SEP3, student investigations must provide evidence to support explanations and include some design features that make a strong investigation. Design features include identification of variables (independent and dependent), the use of an appropriate control, identification of the tools needed to gather data (ex. plots and transects), identification of how measurements are to be recorded, and how much data are needed to support a claim (NGSS Appendix F, 6-8th elements 1).
● Casual or unrealistic experiments not grounded in a scientific rationale should not be coded as SEP3.
● Observations count as part of carrying out an investigation (observational study) when they are made in order to answer a scientific question. Observations where students are just identifying patterns or trends would not be coded as SEP3, but may lead to generation of questions and coding in SEP1. Be careful not to code observations that are starting points to generate questions.
● SEP3 vs. SEP6: Do not code SEP3 if students are not designing their own investigation. If students are critiquing an experiment in light of their explanation or argument then code it in SEP6 or SEP7. If they are planning a future investigation based off the design weaker previous experiment.
● SEP3 vs. SEP4: It counts as 3 when they are recording data during an investigation in a table, and it counts as 4 when they are organizing data for interpretation.
Observing any of the following would be coded for SEP3. Students working collaboratively or individually:
⬜ plan an investigation. See elements of planning an investigation above.
⬜ evaluate the strength of an investigation. This includes general discussions of design features of a strong investigation.
⬜ make predictions about what they will observe based on their own investigation.
⬜ carry out an investigation (including observational studies).
SEP4: Analyzing & Interpreting Data / Students analyze data to interpret the meaning of the data.
Note:
● Collecting data does not count for SEP 4. This should be coded as SEP3, as an aspect of conducting an investigation.
● We will see students interpret and analyze data in order to make an evidence-based claim. SEP4 should be checked when students are at the initial analysis and interpretation of data, but have not yet connected the data back to the scientific question, hypothesis, or reasoning to construct an explanation. If students are using data in order to support a claim/explanation, this should be coded as SEP6.
Observing any of the following would be coded for SEP 4. Students working collaboratively or individually:
⬜ organize data or construct graphs.
⬜ discuss or compare different data representations.
⬜ identify, analyze, or interpret trends/relationships in graphs or data. BSCS I2 tool. This includes exploring large datasets using computers to identify patterns and trends.
⬜ discuss variation in data.
⬜ discuss strengths or limitations in data. This would include discussing causal vs. correlational relationships; measurement error, variables appropriately represent system/question, reliability of data source...).
⬜ apply concepts of statistics and probability to analyze data. If students calculate statistics and probability and then use them to interpret data, code as SEP4 and SEP5.
SEP5: Using Mathematics & Computational Thinking / Students are expected to use mathematics to represent physical variables and their relationships. Applications of mathematics in science include logic, geometry, calculus, statistics, probability, and mathematical functions. Students are expected to engage in computational thinking, which involves strategies for organizing and searching data, and using and developing new simulations of natural systems (NGSS Appendix F). Students use mathematics to transform and organize data (ISIOP Item 22).
Observing any of the following would be coded for SEP 5. Students working collaboratively or individually:
⬜ perform calculations. This includes calculations of ratios, rates, percentages, algebra, equations, formulas, functions, etc. (NGSS Elements 3 & 5).
⬜ summarize data with statistics. Calculate means, range, standard deviation, standard error, regression, etc.
SEP6: Constructing Explanations and Designing Solutions / Students make claims and support them with evidence and reasoning as they construct an explanation of a phenomenon.
Note:
● A summative, formal presentation or written explanation for a phenomenon should be coded in SEP 8, Obtaining, Evaluating, and Communicating Information, not here. Conversely, instances of students working to develop an explanation should be coded here.
● Students may be examining evidence and data while constructing a claim. When students are selecting appropriate data or evidence to support their explanations, SEP6 should be coded. If interpretation and analysis of data is not in service of evidence for a claim, code SEP4.
● Explanations should include qualitative or quantitative relationships between variables (NGSS element 1) and be based on valid and reliable evidence/data (element 3).
● We define a claim as a statement that can be supported by evidence, linked through scientific reasoning. The claim can be in response to a posed question, or an assertion of the truth of something if no formal question has been posed.
● A claim and hypothesis can potentially overlap (code SEP6 and SEP1). However, a claim is more narrow than the hypothesis and can be fully supported by the data in hand. A claim may pertain to a particular investigation and not a more general phenomena.
Observing any of the following would be coded for SEP 6. Students working collaboratively or individually:
⬜ construct written explanations:
○ make a scientific claim
○ support claim using data as evidence
○ reasoning: apply scientific ideas or principles to explanation
⬜ construct oral explanations:
○ make a scientific claim
○ support claim using data as evidence
○ reasoning: apply scientific ideas or principles to explanation
SEP7: Engaging in Argument from Evidence / Students critique competing or currently accepted explanations. Includes constructing, using, and discussing oral and/or written arguments. Like SEP6, an argument relies on the CER framework, however the distinction is that the explanation is constructed to either support or refute an existing explanation (6-8th element 3).
Observing any of the following would be coded for SEP 7. Students working collaboratively or individually:
⬜ critique arguments based on the strength of the evidence and reasoning behind the argument.
⬜ revise an existing explanation in the light of new evidence or data (ISIOP).
SEP8: Obtaining, Evaluating, and Communicating Information / Students demonstrate facility in obtaining information, evaluating the appropriateness or validity of information, and communicating information. Students should be able to read, interpret, and produce scientific text (NGSS Appendix F). There are many ways that students can present and communicate their work, including orally, in writing, or visually.
Observing any of the following would be coded for SEP 8. Students working collaboratively or individually:
⬜ obtain information from other sources.
⬜ evaluate the sources of data and facts.
⬜ critically read scientific text converted for classroom use.
⬜ develop and/or make a summative, formal written or oral presentation report that explains a phenomenon, the results of their investigation, etc.
⬜ share or paraphrase a summary of science literature they have read, their scientific explanations or arguments, or their results or graphs. NGSS (Appendix B, p. 64): “Communicating information, evidence, and ideas can be done in multiple ways: using tables, diagrams, graphs, models, interactive displays, and equations as well as orally, in writing, and thorough extended discussions”.
To distinguish this practice from SEP6 or 7, think about the purpose of the activity. Is it to generate the ideas behind the explanation, or is it to communicate existing ideas to others?
Notes:
● How to distinguish SEP7 from SEP8 (see final check box above)
*Although the SEPs are described in terms of what students are doing, we count this as indicative of the teacher’s selection of activities and creation of a classroom culture to support development of these skills.