Protocol for the Baseline and Posttest Strategies for Success Instrument (BP-SFS): Direct Services
Fiscal Year 2017
Written by:
Pacific Institute for Research and Evaluation
July 2017
For The New Mexico Office of Substance Abuse Prevention
For questions about this survey or its administration please contact:
Martha Waller, Ph.D.,
Liz Lilliott, Ph.D,
Lei Zhang, Ph.D,
Christina Lopez-Gutierrez,
Table of Contents
Chapter 1:Introduction
Background information on the Baseline and Post Test Strategies for Success (BP SFS)
The ATOD Use Tool
The Violence Tool
The Resiliency Tool
Getting Parental Consent
Getting Student Assent
When to Conduct the Pre-test
When to Conduct the Post-test
Who Should Conduct Surveys?
Data Entry
When and To Whom to Send Data
Chapter 2: Detailed Instructions for Administering Surveys
Sections of the Strategies for Success Instrument:
Some Basic Instructions:
Basic protocol:
Script for ATOD Use Modules:
Points for Survey Administrators:
Frequently asked questions for survey takers and survey administrators:
Possible Questions about the Drug use Questions:
Additional script for Violence Questions
Possible Questions about the Violence Questions:
Additional script for Resiliency Questions
Important Note on the External Resiliency Module
Possible Questions about the Resiliency Questions:
Chapter 1:Introduction
Background information on the Baseline and Post Test Strategies for Success (BP SFS)
There is one main module inBP-SFS:
The ATOD Use Tool (Module A): 1)A middle school version & 2) a high school version
You can access all these documents and more at the NM Prevention website ( or
The Baseline and Posttest ATOD Use Tool is REQUIRED
For OSAP Direct Service prevention providers.
The ATOD Use Tool
Programs will use the appropriate age group tool for their respondents. If you are working with high school students (9th -12th grades) please use the high school version. If you are working with middle school/junior high students (5th – 8th grades) please use the middle school version of the tool. If you are working with a mix of middle school and high school students, please use the questionnaires that correspond to their grade level. NOTE: These questions match some questions from the YRRS almost identically. You are required to use this tool if you are implementing prevention programming to students directly.
Getting Parental Consent
Before implementing a direct service prevention program with minors you must have parental consent for the student to participate. This process will require getting consent to participate in the program as well as to participate in data collection. You will need to work with the school administration or program administrators to determine the best way to get parental consent and follow their guidelines and requirements in doing so. It may be that you are required to obtain written parental consent on multiple aspects of the prevention program that include data collection. PIRE can assist you with developing a parental consent form in both English and in Spanish and with working with administrators to determine the best way to get parental consent.
Getting Student Assent
Students are not required to participate in the data collection process even if their parent(s) consents for them to do so. They must be informed of their right not to participate and their right to skip questions they do not want to answer on the survey. Later in this document we will provide you with some language to help explain this to your students. You must explain to the students their own rights to participate or not in the data collection process.
When to Conduct the Pre-test
Because these instruments ask fairly personal questions about risk behavior, we recommend that you do not administer the pre-test on the first day of the program. Depending on how often you meet with the students, you may want to wait a week or two until some trust has been established between the adult providers and the students before giving the pretest. You will want to discuss with your evaluator about the best time to give the pre-test. This will likely depend, in part, on the program being implemented and the participants.
The surveys will be available in both pdf (not modifiable) and Word documents (modifiable). We suggest printing from the pdf version as to avoid formatting changes that can occur when printing from Word and just print only 1 cover page.
When to Conduct the Post-test
The post-test data ideally will be collected at, or very near the end, of your program. Note that the post-test includes additional questions to the pre-test.
Who Should Conduct Surveys?
The local evaluator(or her team member) will be responsible for collecting the data. This is to ensure that neither the program providers nor any school personnel know the participants’ unique identifiers and risk the loss of the confidentiality of the participants’ responses. The participants must not feel that their instructors will know their responses.
The local evaluator should complete the cover page of the survey at BASELINE and at POST intervention and should complete the ID number on the first page of the survey. This should not be done by the provider.
The local evaluator will collect the cover pages with the information linking the student’s name to their unique identifying number in a folder or envelope. The cover page documents will be stored by the local evaluator and not the provider.
After the local evaluator has entered them into the data file and all data have been cleaned and verified the local evaluator must mail the paper surveys (without the coversheets) to:
Dr. Liz Lilliott
BHRCS
612 Encino Place NE
Albuquerque, NM 87102
PIRE will store the paper surveys for 5 years before shredding the documents.
After the administration of the survey a debriefing should be conducted to assess whether there were any problems encountered while giving the survey such as questions that were unclear, reading comprehensions issues, classroom disruptions, breeches of confidentiality, etc.
Data Entry
Data entry should be completed by the local evaluator and not the provider to protect the privacy of the students. Data should be entered into the provided data entry templates and submitted to PIRE in a timely manner. Official data entry trainings will take place but some basic instructions follow.
Instructions
1.Variable name convention: Each item in the survey corresponds to a variable name in the data entry file and is differentiated by its module and the timing of tests (i.e., pre-test vs. posttest). Each item number is preceded by the letter “B” (baseline) or “P” (posttest), then followed by a module indicator (e.g., Module A). Therefore, variable name for item number 5 in Module A at baseline would be ‘BA05’. Please note that the variable name is in two-digit number format.
2.In most instances, the option numbers below each item indicate value for that item, unless otherwise instructed. Therefore, for item #1, if the respondent indicated that he is a male and circled ‘1’, then ‘1’ would be entered into the data file.
3.Other special instructions for the particular item are indicated in italics and bold in the code book.
4.For missing data, please leave it blank. Do not enter ‘0’, ‘99’, or any other codes.
5.For the ID #, we have provided 2 spaces for group ID. You may use this space for two purposes. First, if you are collecting your own comparison data, you may use the group ID to distinguish between target and comparison groups. If you are not collecting your own comparison group data, you may also use the group ID to distinguish between program providers/instructors. This could allow you to determine if different instructors have different outcomes. Used as an educational analysis, a program can see if some instructors need booster training sessions on some aspects of the curriculum or examine whether the sex of the instructor differentially influences outcomes, etc. If you do not have multiple groups or instructors and do not need the group ID, please enter: 00 in those two places.
When and To Whom to Send Data
When: Any existing pre- and/or post-test data should be sent to PIRE by January 15th, 2017. Any remaining pre- and/or post-test data should be sent to PIRE by June 15th, 2017. All paper surveys- BASELINE and POST should be sent to PIRE by August 30, 2017.
Who: Electronic Data files should be sent to: Lei Zhang at: and please cc Martha Waller at: and your program manager.
All paper surveys (without the cover sheets) should be sent to:
Dr. Liz Lilliott
BHRCS
612 Encino Place NE
Albuquerque, NM 87102
Any data that are sent to PIRE at pre-test will be expected at post-test unless we are informed otherwise.
Chapter 2: Detailed Instructions for Administering Surveys
Sections of the Strategies for Success Instrument:
1)BASELINE SFS: ATOD Use - Middle & High School (REQUIRED)
2)POST SFS: ATOD Use – Middle & High School (REQUIRED)
Some Basic Instructions:
MAINTAINING THE CONFIDENTIALITY OF THE STUDENTS’ RESPONSES IS CRITICALLY IMPORTANT.You will need to match pre- & post-intervention data by respondent. Therefore, the local evaluator should have a master list (that is private and not shared with the provider) that assigns each participant a unique ID number. The respondents will be given surveys with their assigned ID number & name on the cover page. The local evaluator will complete this information and hand out the surveys to the students. This will make sure that each student receives the correct survey. The student should tear off the cover page and turn it in to the local evaluator prior to taking the survey. The survey itself will have only the ID number on the top of the first page but not any name. This will help provide some assurance to the students that the responses will remain confidential and their instructors and any school personnel will not know their responses. The local evaluator is the only individual who should have access to the list with the names and corresponding ID numbers. The local evaluator should keep the cover pages from the surveys in a safe and secure place. These should not be stored with the prevention provider.
Some prevention programs may work with a small number of participants. The risk for disclosure of the respondent’s identify increases when there are few participants. Therefore, PIRE will store all paper surveys (but not the cover pages) in a secure location. Paper surveys should be mailed to the Albuquerque office where they will be stored for 5 years before being shredded. This will ensure that no one at the provider level will know of an individual’s responses.
Basic protocol:
1)The Prevention Provider must obtain parental consent and student assent to participate in the data collection process.
2)The local evaluator will complete the information on the cover page and the first page of all the surveys to be administered at BASELINE and POST intervention.
3)The local evaluator will distribute the surveys to the student respondents.
4)The local evaluator will explain the rights of the student respondents to not participate or refuse to answer any question. (See sample text below.)
5)The local evaluator will explain how the student respondents’ confidentiality of responses will be maintained. (See sample text below.)
6)The local evaluator will ask the students to tear off the cover sheet from the survey and collect the cover sheets. The local evaluator will securely store the cover sheets in their locked files.
7)The local evaluator will administer the survey and collect the completed surveys from the students. (The provider, prevention staff, and school staff should not administer or collect completed surveys from the students to help protect the confidentiality of the students’ responses.)
8)The local evaluator will debrief with the student participants. Provide the take home information to the students at both BASELINE and POST data collection.
9)The local evaluator will debrief with the provider about the process.(Ask about translations if appropriate, and questions that came up, concerns, etc. Share with PIRE if corrected are needed to the survey or to the data collection process.)
10)The local evaluator will enter the data into a designated electronic data file.
11)The local evaluator will mail the paper surveys (without the cover sheet) to PIRE whowill store all the paper surveys securely. The local evaluator will store all cover sheets. These must be kept separately.
12)The local evaluator will send the electronic data files to PIRE.
Script for ATOD Use Modules:
Note to survey administrators: feel free to place the script below in your own words. It consists of the primary points for informed consent so it is important to address each one.
- First I want to explain why we are taking these surveys.
- Manyyoung people in New Mexico participate in prevention programs like this one. It’s important that you take this survey so we can learn how well these programs work.
- Program staff will determine whether this prevention program works or not by asking each student to do a survey at the beginning of the program and one at the end of the program.
- You are not required to complete this survey. You have the right to refuse to answer any question on the survey. This survey is for evaluation purposes only and it not intended to assess any one person’s behavior but rather as a group.
- This is NOT a test. There are no right or wrong answers and we want you to try to answer all the questions. You may feel some questions do not apply to you. And sometimes you may be asked what seems like the same thing more than once. But we still like for you to try to answer every question by selecting the best answer of those provided.
- You’ll notice on the front page of the questionnaire we assign you a number so nobody knows who you are. This is very important for you to understand. None of yourteachers, your parents or guardians, and no staff person in the program will ever know what you say on this survey. All anyone you know who looks at responses will ever see is a number and not a name that goes with the responses. This way we can help guarantee you that your answers will remain confidential. We want you to answer as truthfully and honestly as you can.
- Because we need to match your responses from the beginning of the program with those at the end of the program, we keep one document that has your name and your ID number on it, which is the front page. At this time, please tear off the cover sheet that has your name and ID number on it now and place in the envelope that is being passed around. I will keep these separate from your survey so your answers remain confidential. As the local evaluator, only I will know this information and I will not share this information with anyone else. Your survey will be stored separately from the cover page. The cover page document will be kept in a locked file cabinet with the evaluator, while the surveys that do not have your name on them, will be kept by an organization called PIRE. At the end of this program, the evaluator will shred the cover pages that connect your name and the ID number. After 5 years, PIRE will shred all the surveys according to the law. No one running this program will ever know your responses.
- Some questions may seem like they are assuming that you are already using drugs, smoking or drinking. If you have never used any of these things, just check off “0”or “never” in the answer.
- Some of these questions may seem like they are asking the same thing, or they may not feel applicable to you. We hope that you can be patient and understand that those who pay for this program need us to ask you these questions in all these ways.
- We need you to be honest in your answers about your own experiences, thoughts and feelings when answering these questions. You will be asked questions about yourself and those close to you, but your individual answers are kept private.
- (If post survey): Try to answer the questions according to how you feel now. It’s okay if your answers are not the same as they were in the first survey you took.
- Do not write your name anywhere on the survey booklet with the exception of the cover page that you have turned in. Erase your name on the booklet if it is written there already. Mark your answers clearly on the sheet. If you change your mind about an answer, erase your first answer completely.
- Please respond to the questions quietly and privately. Respect your neighbor and do not share or talk with them while you are doing this short survey.
- If you have a question, feel free to raise your hand so I can respond to your questions individually.
Points for Survey Administrators: