Society for Prevention Research 2009 Cup Winners from PSU

Society for Prevention Research 2009 Cup Winners from PSU

with Beau Abar, Melissa Lippold, and C.J. Powers

June 4, 2010

HostMichael Clevelandinterviews Beau Abar, Melissa Lippold, and C.J. Powers, members of the winning team of the 2009 Sloboda and Bukoski SPR Cup competition.

Announcer:Methodology Minutes is brought to you by the Methodology Center at Penn State, your source for cutting edge research methodology in the social, behavioral, and health sciences.

Michael:Welcome to Methodology Minutes. Today I am interviewing three of the team members from the 2009 SPR Cup Competition. These members are Beau Abar, Melissa Lippold, and CJ Powers. Welcome to all three of you.

Beau:Thank you.

CJ:Thank you.

Melissa:Thanks.

Michael:OK, and two other members of the 2009 Cup team were not able to attend today, but I would like to mention their names. They are Caitlin Abar, a Ph.D. candidate in HDFS, and Elizabeth Manning, also a Ph.D. candidate in HDFS. I will start by giving you a brief opportunity to introduce yourself. Beau, you are a graduate student in HDFS?

Beau:Yeah, I'm a fifth-year Ph.D. student just about to graduate.

Michael:Very good. Melissa, you are also a HDFS graduate student?

Melissa:Absolutely. In my fourth year.

Michael:CJ, you are in Child and Clinical Psychology?

CJ:That's correct. Yes, I'm a fourth-year Child Clinical student.

Michael:Very good, thank you for all of you being here today and joining us for this edition of Methodology Minutes. I want to start by introducing our listeners to what the SPR Cup is all about. Most people I am assuming will know that SPR stands for the Society for Prevention Research. Hopefully, most of our listeners are also aware of the SPR Cup, but I did copy down the words from the SPR website that describes what the SPR Cup is all about, so let me just read that.

It says, "Each year at the annual conference, SPR sponsors a competition among teams of researchers for the traveling Sloboda and Bukoski SPR Cup. The Cup is named for two of the founders and longtime active members of SPR, Dr. Zili Sloboda and Dr. William Bukoski. The SPR Cup competition is an opportunity for an unique experience: several independent teams of scientists, each working with the same data set, problem solve together for a brief period of time and then jointly present their ideas to each other and then to the larger group of experienced prevention scientists at the SPR Conference."

We know a little bit about that from that description, but can you as team members maybe describe a bit more in detail about what happens when a team member or a team signs up to compete in the SPR Cup.

Beau:You sign up as a team and you get in contact with program director and a couple weeks later they send you a dataset and then in about six weeks before the conference and you're told to provide the best paper that you can in that six weeks that's as prevention-focused as possible given your data.

Michael:Right.

Beau:Before the competition, you really have no idea what type of data you're going to get.

Michael:When you said they send you a dataset, you don't know what dataset you're going to be working with.

Beau:We had no idea whatsoever.

Michael:OK.

Beau:Each of the past several years that it's been done, Penn State has had a team and it's been a different dataset each time, so it's always kind of a surprise when you first get it.

CJ:Even before we received the dataset, I think we talked and met up and decided we were going to be a team, put our name in, and then because we knew it was a quick timeline, so the time between when you get the dataset and when you actually have to present it is about six weeks. We met before that to go over what our strengths and weaknesses were, and divide up what we thought we were going to attempt to tackle ahead of time, and how we would work as a team.

Melissa:We put a lot of thought even in actually creating the team, what do each of us bring to the table, what are our strengths.

Michael:Oh really?

Melissa:We were really ready when the data came to hit the ground running and had some ideas about who would do some of the background and conceptual work, who would start on the data, get us up to speed on that, and we had an outline of plan. But as soon as we got the data, I think the first thing we did was start reading. We put some time into looking at what is this dataset, what has been done with this dataset, and what's there for us, to get a little bit of a background before we started brainstorming specific ideas to use from it. But we were a little surprised, I think, that the dataset was focused around education, which was different than past years that more of a substance use or mental health perspective.

Michael:Sure.

Melissa:There was definitely some getting up to speed on our end about that topic, and how we might think about that from a prevention point of view. I think an exciting thing about that was to do work in area that I wasn't particularly fluent in initially.

Beau:I think actually that was one of the challenges that it wasn't really a substantive strength of any of ours. We're trying to figure out what are the questions in this field. They clearly linked it with ongoing policy implications at that time, so with Obama's stimulus plan and focus on education, so we looked at what that plan was, and tried to come up with a question that might fit in with and had a clear policy implication, to make it more interesting to a general audience as well.

Michael:OK.

CJ:The dataset that they gave us was the educational longitudinal study of 2002, and the dataset was really expansive. It had student reports, parent reports, teacher reports, administrator reports.

Michael:Wow.

CJ:I think it about 17,000 students included across 150 or so schools across the country.

Michael:Right.

CJ:It was very expansive, but as Melissa said, it was focused on education, which wasn't a substantive strength of any of ours to a great degree going in.

Michael:Yes. You talked a little bit about before you had the dataset you had divided you responsibilities. Someone said I'm strong in methods or I'm good at this area, and so you had some sense of that. Did those play out and actually correspond to what happened when you got the data? Did those strengths and weaknesses correspond to the dataset that you received because of the difference in terms of educational data versus

CJ:I think the way we split up the tasks fell out along that. I know Beau and I were the more methodology-focused people, so we looked closely at what variables were there and how we would operationalize what constructs we wound up using. I know Melissa and Caitlin and Liz had strengths and were more conceptually interested and so did a lot of the background research and the lit review and thought about implications and were focused around what interesting questions are out there.

Michael:OK. That process was taking place simultaneously? Where you, CJ, and Beau were looking through the dataset, the variables, and Melissa and others were looking through the literature and developing questions, it sounds like?

Melissa:Yeah, we brainstormed questions together, but thinking through, once we had our question, there was a lot of thought about how does this question fit into this bigger picture, and conceptually how do we think this works. What do we think are the underlying mechanisms? It took a lot of thinking that through in part because we'd all had some experience with education work, but it wasn't an area of expertise and so there's a lot of thought that went into framing that and how to make it relevant, because that was something that we were all really interested in. We wanted this to be a paper that would be useful to people in the education field, thinking about how to improve prevention programs or integrate prevention.

Michael:Not just an exercise for an SPR Cup.

Melissa:Right, exactly.

Michael:A bigger purpose in mind behind it.

Melissa:Exactly.

Beau:The division of tasks that we were talking about, one thing that made our group effective was the fact that even the individuals that were focused on the conceptual issues were very methodologically talented, so that things that CJ and I would bring to a meeting, they were very aware of the methods that we were using. They were very aware of how to critique them and to improve upon them and the conceptual and theoretical questions and tie-ins that they would be bring to the meetings CJ and I were capable of talking about and batting back and forth. I think that was a real strength of our group. Instead of just being the numbers people and the words people, we really were a mixture.

Melissa:We met regularly, at least every week, maybe even more often during the peak time. There was a lot of team work, helping each other out during certain segments. I think a lot of it was definitely a group think process, with different people taking the lead on different parts of it. I think it ended up feeling like it was a cohesive project or cohesive thing at the end.

Beau:I agree.

Michael:Had any of you worked together before?

Beau:I had worked with my wife, Caitlin, on several projects, and I had TA'd with Liz Manning on some of the projects.

Melissa:Yeah, CJ and I had not officially worked together, but we were in a lot of statistics classes together. Pretty early on actually in my graduate student career CJ and I identified each other as stats buddies and since SPR have worked on some things together.

Michael:Oh, nice.

Melissa:We had some experience working together and collaborating. Beau and Caitlin and I were all in the PAMT program, the Prevention And Methodology Training program.

Michael:Right.

Melissa:We had some experience even though we hadn't worked on a particular paper of thinking through things together and giving feedback on each other's work and that type of thing just from being part of that community together.

Michael:Sure. The SPR Cup was a good opportunity for you to maybe actually explicitly work together on things that things maybe you talked about in the past, like we should do something together. We should collaborate or something like that so it was a good opportunity.

Beau:Also, one of the things that I enjoyed about the Cup is that it was the first time where all of us had a chance to collaborate with a group of peers. It's different when you're working under a mentor or something, and they're the go to person to talk to when have any questions, and in this case, while we did have some people that we talked with, it was really run by us.

Michael:Yes.

Beau:I think that's a different type of experience when you're working out who's going to do what. Even authorship order and some of the research issues that often get worked out for you we then worked out amongst ourselves, and I think that was a really good training experience as well.

Michael:Absolutely. I think we've alluded to it, but I'd like to hear a little bit more about the results and your questions that you chose to address in this study.

CJ:One thing we were really interested in and what the ELS 2002 really discussed in some depth was high school dropout. There was a big push at the time and there continues to be a big push by legislatures trying to come up with ways to fund programs that keep kids in school.

Michael:Right.

CJ:That was something that jumped to all of us as we were looking through the data as something we can really focus on prevention-wise.

Michael:Right.

CJ:What we wanted to do is to try to figure out beforehand, before dropout occurred, what types of motivations do kids have for going to school. Why are they going to school?

Michael:OK.

CJ:We had dropout statistics at 10th grade and we had motivations for attending school. Pardon me, we had dropout at 12th grade and we had motivations for attending school at 10th grade.

Michael:OK.

CJ:We had a nice ability to do some prospective analysis. We used latent class analysis on the motivations to see if there were different types of motivations, different clusters of motivations that existed within this high school population, and we found five different classes of motivation. There were some kids who were motivated for academic reasons, job reasons, and social reasons, like seeing friends. There were others that were just focused on job and social reasons. I need an education to get a good job later and I like to see my friends.

Michael:OK.

CJ:Then there was a group who was focused on academics and jobs later.

Michael:Yes.

CJ:Then there were two smaller groups who were either focused solely on job-related reasons for attending school or focused solely on social reasons. We had these motivation classes so we used them to predicate 12th grade dropout while accounting for their 10th grade achievement, knowing that your achievement in 10th grade is going to have a strong association with whether or not you chose to drop out at 12th grade.

Michael:Sure.

CJ:We included these and we found that motivational class predicted, even when accounting for previous achievement, such that not surprisingly the kids who were only motivated for social reasons for attending school tended to drop out at the greatest rates.

Michael:They were the more likely to drop out, that class.

CJ:Yes, what was a little bit surprising were those individuals who were solely motivated for job reasons, they also were at risk for elevated dropout, which was somewhat surprising given that their primary motivation is for improved job status later, they still dropped out at much higher rates than the three other classes.

Following these findings, we wanted to know potential ways that we can actually influence these motivations, potential predictors of these motivations as targets of later interventions.

Michael:Right.

CJ:To do that, we looked at a host of parent-related characteristics.

Beau:Actually, can I interrupt real quick?

CJ:Yes.

Beau:I just wanted to talk a little bit about I think one of the things that was unique about the latent class approach too is that it's different from a dimensional approach so if we were just looking at types of motivation and doing more of like a horse race type analysis of the social motivation more important than job motivation or even in combinations, we had seven different types of motivation and so the number of interactions you can look at and the combinations was just too much.

Michael:It would be impossible to model all of that.

Beau:What we were interested in is not so much which motivation is best, but are there profiles of motivation that are really important, and it turns out that that's true, that basically it fell roughly in line with the more reasons you have for being in school, the better off you're going to be. You're more likely to graduate. But with the latent class got us more than just saying one motivation versus two motivations versus three motivations. Now we know there are certain types of motivation where you're not likely just have academic motivation but not also be interested in social and job. We can also find that if you're just interested in job motivation or just interested in social motivation, that's a high risk category, and what's the thing that you're most likely to, what would you want to target in each of those in order to bump you into the next lowest level of risk.

Michael:Oh, OK.

Beau:There was no social and academic motivation profile, so maybe at that point if you're just interested in the social aspects of school you wouldn't necessarily do an intervention to target academics. But if you did target not only is school a fun place to have friends, but also a good place to get an education to help you get a job, that that class actually exists, and is likely to lower your risk of dropout.

Melissa:We were pretty excited by it. We did some preliminary analysis. First, we had this idea and we didn't know what patterns of motivations would come out of the data. Latent class is kind of inductive in that way. We were pretty excited just by the patterns of motivations and styles before we even looked at what predicts these motivations themselves. We felt like thinking about these motivations holistically or looking at the patterns themselves produced some interesting findings, and then actually got quite excited when they were so strongly linked to dropout. They had a really strong relationship, even when you account for achievement.