Accepted by Science Communication on 27 August 2012

POST PRINT VERSION.

Accepted by Science Communication on 27 August 2012.

*Note – this is a copy of the final draft version submitted on 5 April 2013 after peer review.

Science Communication (2012) 35(6) 780-809

DOI: 10.1177/1075547013491398

Author details:

Corresponding author: Adrian Cherney

*Dr Adrian Cherney

School of Social Science

The University of Queensland

Brisbane, St Lucia 4072

ph + 61 7 3365 3236

fax + 61 7 3365 1544

email:

Dr Jenny Povey

Professor Brian Head

Professor Paul Boreham

Michele Ferguson

Institute for Social Science Research

The University of Queensland

Brisbane, St Lucia.

Acknowledgements: This project is supported through ARC Linkage project: LP100100380.

Research Utilization in the Social Sciences: A Comparison of Five Academic Disciplines in Australia

Abstract

Social science disciplines generate diverse forms of research utilization, given the various contexts in which disciplinary knowledge is produced and translated for the fields of policy and practice. We examine this issue from the perspective of academic researchers in the social sciences across education, economics, sociology, political science and psychology. We use survey data from a study of university-based social science researchers in Australia to examine factors that influence perceptions of the policy uptake of social research.Our results show that disciplinary and methodological context matters when it comes to understanding the translation, dissemination and utilization of academic social research.

Keywords: research utilization, translation, research impact, social science, research collaborations.

Introduction

The need to improve the dissemination and translation of social research for non-academic audiences, and to increase theimpact of academic research has gained increasing attention across a range of academic disciplines. These include social work (Wilkinson, Gallagher and Smith 2012), education (Rickinson, Sebba & Edwards 2011), economics (Banks 2011), sociology (Cherney and McGee 2011), political science (Rogers 1989),psychology (Meagher, Lyall and Nutley 2008) and public health (Contrandriopoulos et al 2010; Haynes et al 2011; Lavis et al 2003, 2006).There have been many recent attempts to establish new processes to promote interaction between social scientists and government and community stakeholders, such as the Dutch ‘science shops’ concept that has been supported by the European Commission and emerged in many countries under various guises (European Commission 2003).

Interest has also been fuelled by government approaches to the assessment of academic research quality,such as the Excellence in Research Australia (ERA) initiativeand the UK government’s Research Excellence Framework. The impact of academic research has also been raised in the United States in the context of how the National Science Foundationfunds research and requires information about relevance for non-science stakeholders(Holbrook 2012; Kamenetzky 2013;Mervis 2011). The issue has gained public attention following calls by some politiciansto reduce social science researchfunding due to its lack of perceived relevance by comparison with medical research (Atran 2013).These issues have coincided with recent attempts in Australia, Europe and the US to gauge the social and economic impact of university research (Allen Consulting 2005; Holbrook 2012; Kamenetzky 2013;Macintyre 2010; Mervis 2011; Juhlin, Tang and Molas-Gallart 2012; Smith, Ward and House 2011; Wooding et al 2007). The issue of research impact has also been emphasized in wider community and industryconcerns that academicsneed to engage more with end-users, a major criticism being that there isa disjunction between research conducted by academics and its uptake by public or private sector agencies(Burkhardt & Schoenfeld 2003; Bogenschneider and Corbett, 2010; Rickinson, Sebba and Edwards 2011; Lambert 2003; for specific Australian commentary see Macintyre 2010; Ross 2011; Shergold 2011).Better channels for communication and knowledge translation seem to be essential (Lomas 2000). Closer synergies such as collaborative research partnerships are seen as a way to enhance the impact of academic social research (Huberman, 1990; Lavis et al 2006b; Nutley, Walter and Davies 2007; Orr and Bennett 2012).

It needs to be recognized that different social science disciplines will potentially generate diverse forms of research use. For instance, research uptake will be influenced by discipline-based contextual factors,which will shape knowledge translationactivities (Landry, Amara and Lamari 2001b; Levin 2011). These contextual factorsinclude attitudes among academics about the value of conducting applied research and about the importance of investing in processes (e.g. partnerships) that help generate research uptake by end-users. This observation draws attention to two related issues: which factors appear to influence social research use, and how these factors may vary across social science disciplines? There is little empirical work on this topic internationally. Investigations of research utilisation have mainly focused on case studies of specific policy domains (e.g. Kramer and Wells 2005; Wilkinson, Gallagher and Smith 2012; Weiss and Bucuvalas 1980) and only a few studies have examined research impact across different research disciplines (seeLandry, Amara and Lamari 2001b). While case studies on research utilization have been important in providing insight into the nuances of knowledge translation and impact,their generalizabilityto other fields is open to debate (Dunn, Dukes and Cahill 1984;Landry, Amara and Lamari 2001a, 2001b; Seidal 1981). The absence of studies that aim to examine research utilization across the social sciencesmeans that our understanding about disciplinary variations in research translation and uptake is limited. Such studies would help shed light on the potential practices or processes that hinder and facilitate research impact and knowledge transfer.

In this paper we examine the issue of research utilization from the perspective of social science knowledge producers – academic researchers in the social sciences across education, economics, sociology, political science and psychology. We use survey data from a study of university-based social science researchers in Australia to examine perceptions of the policy uptake of social research. Our broader aim is to better understand factors that facilitate the utilization of social research by non-academic audiences – whom we have termed end-users[i]. Importantly we advance understanding in the field of knowledge transfer and use by examining how research utilisation varies across social science disciplines, accounting for any major differences. Our findings also have wider implications for evaluating research impact in the social sciences as well as broader lessons about how research is communicated to end-users to improve research uptake.

.

Background and literature review

Conceptualising Research Utilization

Understanding the impact of social science research has been a primary focus of research utilization scholarship which, since the 1970s, has examined the factors and circumstances which support or undermine the uptake of social research by policy decision-makers and practitioners (Caplan 1979; Larson 1980; Lester and Wilds 1990; Rich 1997; Weiss 1980; Weiss and Bucuvalas 1980). It is a field not only concerned with the behaviours and decisions of those whoconsume social research but also with investigating how the circumstances and activities of those whoproduce social research (e.g. academics, research institutes or private think tanks) influence processes of knowledge transfer and uptake (Cherney et al 2012a, 2012b; Florio and Demartini 1993; Hayes et al 2011; Landry, Amara and Lamari 2001b; Lavis et al 2003; Weber 1986; Weiss and Bucuvalas 1980).

When it comes to measuring research utilization, no singleconceptual model has gained unanimous approval (Belkhodja2012; Rich 1997). One reason for this is the methodological problem associated with specifying the dependent variable of research use, given that it can be defined either as a process or an outcome (Rich 1997). From one perspective, research utilizationcan be viewed as a definitive end-point where research has a direct impact on policy or practice.This is often referred to as the problem-solving model,with research seen as inducing changes in policy decision-making (Belkhodja 2012; Weiss 1980). However this view has been criticized as misplaced because it ignores the non-instrumental forms of research impact, such as conceptual or symbolic forms of utilization, which involve research being used to change understanding (i.e. conceptual use) or to confirm and promote pre-existing policy directions or commitments(i.e. symbolic use) (Amara, Ouimet and Landry 2004; Belkhodja 2012; Lavis et al 2003; Weiss 1980). It has also been argued thatresearch utilization rarely follows a linear path from academic knowledge producers to end-users in fields of policy and practice and hence uptake of research evidence may be more diffuse (Juhlin, Tang and Molas-Gallart 2012;Weiss, 1980).

In this study we measure research utilization by adopting a stagesor process model, replicating a modified version of the Knott and Wildavsky (1980) research-use (RU) scale, similar to that used in the study by Landry, Amara and Lamari (2001a, 2001b).This model comprises 6 stages - transmission, cognition, reference, effort, influence and application - and Table 1 provides the descriptions for each stage as presented in our questionnaire to Australian social scientists. The RU scale characterizes research use as both a process and an outcome, in the sense that cognition builds on transmission, reference builds on cognition, effort on reference, influence on effort, and application on influence. The different stages encompass a range of outcomes and reflect an increasing level of knowledge absorption by end-users (Belkhodja 2012). The RU scale does providethe ability to measure the significance of factors that have a bearing on research use,and has beenshown to be a reliable scale (Belkhodja 2012; Cherney and McGee, 2011;Cherney et al 2012a, 2012b, Landry, Amara and Lamari 2001a; 2001b; Lester and Wilds 1990; Lester 1993).

INSERT TABLE 1 HERE

Factors Influencing Research Use

Just as there is no agreed conceptual model relating to research utilization, there is no definitive list of variables developed to help predict knowledge use(Lester, 1993). Singular and multiple perspectives have been proposed. These include, for example, the science-push perspective, which explains advances in research utilization as largely generated by the actions of knowledge producers and the types of products they produce(Belkhodja 2012; Landry, Amara and Lamari 2001a). Alternatively Belkhodja et al (2007) adopt an organizational perspective, focusing on multiple contextual variables to explain knowledge utilization that encompassorganizational interests, the needs and behaviors of researchers and end-users, and levels of interaction between researchers and users (also see Belkhodja 2012). Taking account of various individual and contextual factors, variables that influence the utilization of academic social research can be groupedunder four broad headings relating to the researchers’ and users’ context,the chosendissemination activities,and the interactions between academic researchers and potential end-users.

The researchers context relates to a mix of supply-side variablesthat influence research production. This includes the academic role or position of researchers, such as whether they are in a research-only or a teaching and research position;the types of outputs such as quantitative and qualitative studies;whether research is focused on non-academic users;the importance of different funding sources;success at securing external research grants; and the institutional drivers that influence the motivation to collaborate with external partners (rewards for collaborative research) (Bogenschneider and Corbett 2010; Contrandriopouloset al 2010; Cherney et a 2012a, 2012b; Landry, Amara and Lamari 2001a; 2001b; Jacobson, Butterill andPaula 2004). Disciplinary backgrounds can also be extremely important ininfluencing research utilization, because these can shapebehaviours and views about dissemination and engagement with end-users – matters relevant tothe culture of knowledge production within particular research disciplinesand the forms of interaction and methods of communication adopted (Bogenschneider and Corbett 2010).

The factors related toend-user contexts encompass judgements on the part of policy-makers and practitioners relating to the value placed on the quality of research evidence, itsperceived relevance,and the political and economic feasibility of adopting research findings. Skills to interpret and apply research findings also matter, as does the levelof access to research products such as reports and journals. Added to this are organisational processes such asthe value policy-makers and practitioners place on research evidence. Such factors influence the overall demand for academic research within end-user contexts (Belkhodja et al., 2007; Belkhodja 2012; Contrandriopoulos et al 2010; Nutley, Walter and Davies 2007; Ouimet et al 2009).

Disseminationvariables relate to efforts by researchers to tailor research products (e.g. reports) for the needs of users, and to develop communication strategies targeting particular non-academic audiences (Cherney et al 2012b; Huberman 1990; Mitton et al 2007). The basic argument is that the more researchers invest in dissemination, the more likely research-based knowledge will be adopted (Cherney and McGee 2011; Mitton et al 2007). This includes holding meetings to discuss the scope and results of research projects with specific users or partners, and targeting particular forums such as thosewhere academics report on their research to government committees.

Finally, interaction variables focus on the intensity of the linkages between knowledge producers and potential users or beneficiaries of research. Interactions generallyhelp the process of dissemination but are usuallybased on informal personal contacts and networks between researchers and end-users. The argument is that the more intensive are these linkages, the more likely research uptake will occur (Huberman 1990; Landry, Amara and Lamari 2001a; Lomas 2000; Mitton et al 2007).

Research Design and Methodology

The data used in this research were drawn from a broader study examining evidence-based policy and practice. The project involves 4 phases: (1) a targeted survey of Australian social scientists; (2) a targeted survey of policy personnel; (3) interviews with a selection of academic respondents; and (4) interviews with policy personnel.Results reported in this paper are drawn from the phase 1 survey.The academic survey was partially based on existing items or scales (Bogenschneider and Corbett, 2010; Landry, Amara and Lamari, 2001a, 2001b) but with additional items included to gauge the dynamics of research partnerships. Questions were framed around a number of themes relating to seniority, research discipline, academic position, grant success, main orientation of the respondent’s research, experience of working with external partners, methods of dissemination, perceived barriers to research transfer to end-users, benefits resulting from collaborations with external partners, the challenges of research partnerships, and the use and impact of the research produced by respondents.

The survey was first piloted among Fellows of the Academy of the Social Sciences in Australia (ASSA) in September-October 2010[ii]. Eighty-one surveys were completed, with a response rate of about 17 per cent. There were no significant changes to the survey following the pilot outside of editing some lead-in questions to make them clearer[iii].No scales in the survey were changed. For the main survey, a database was established of Australian academics who had secured at least one Australian Research Council (ARC) grant (known as Discovery or Linkage grants[iv]) between 2001 and 2010 within the field of social and behavioral science[v]. The selection of relevant disciplines from which respondents were sampled was based upon the ‘field of research’ codes used by the ARC to categorise the funded projects, and comprised codes relating to anthropology, criminology and law enforcement, human geography, political science, policy and administration, demography, social work, sociology, other studies in human society, psychology, education and economics. Using this database, a web link to the survey was sent via email to 1,950 academic researchers between November 2010 and February 2011. The same reminder email was sent twice during this period and the survey closed in May 2011. A total of 612 completed surveys were received, which constitutes a response rate of 32 percent. When the main academic survey was combined with the ASSA pilot, the final total was 693 responses (see also Cherney et al 2012b).In this paper we have drawn on results from the same questions used in the pilot and main survey.This final sample includedrespondents from the following main disciplines:education,economics, sociology,political science,and psychology. These disciplines comprised the5 largest discipline clusters in our sample, and will form the primary basis of our analysis. The remaining disciplines have been grouped as ‘other’ (see Table 2)[vi].

INSERT TABLE 2 HERE

Dependent variable

Research utilizationwas measured using a modified version of the Knott and Wildavsky (1980) research use scale, which comprises six stages: transmission, cognition, reference, effort, influence, and application. For each of these six stages, respondents were asked to estimate what had become of their research using a 5-point scale ranging from 1 (never), 2 (rarely), 3 (sometimes), 4 (usually), to 5 (always).

Previous research (Cherney et al 2012a) has shown that ‘failure’ in one stage does not preclude academic researchers from progressing to other stages. Data reported in Table 3,illustrating the proportion of academics who pass or fail at each stage of research utilization for each discipline, indicates that academic social researchers do not necessarily have to traverse in sequence each rung of the research utilization ladder to reach the ultimate stage, namely, the substantive application of research findings by end-users (see Table 3). This finding tends to support arguments relating to the non-linear nature of research transmission, uptake and use. Table 3 illustrates that academic researchers, particularly in political science and psychology, perceive that the uptake of academic research declines in the effort, influence and application stages. In other words there was a decline in the perceived level of influenceof academic research during the process of research utilization by external agencies. Later stages of the RU scale (effort, influence, application –see Table 1) can be particularly challenging for academics to influence directly,given that decisions by policy-makers or practitioners to adopt or apply research evidence can be determined by factors (e.g. political considerations) over which academics have little control.

INSERT TABLE 3 HERE

Figure 1explores the proportion of academic researchers who cumulatively did notpass all six stages of the research utilization scale[vii]. The disciplines of political science and psychology have the highest proportion of researchers who did not perceive their research as being adopted by end-users, whereasacademics in the field of education had the highest number of academics reporting significant utilization by end-users.