Scientometrics - A Study of Scientific Parameters and Metrics
Sudeepa Roy Dey1, Archana Mathur1, Gambhire Swati Sampatrao1, Sandesh Sanjay Gade2, Sai Prasanna M S1
1PES Institute of Technology, Bangalore South Campus, India.
Email: , , ,
2University of Virginia, Email:
ABSTRACT
The term “Scientometrics” emerges from two significant words –Science and Metrics. It is concerned with metrics used for quantitative analysis of researcher’s contribution to various scientific domains. An effective medium to communicate scientific knowledge is via scholarly publications. It provides a platform to propagate research output within and across domains. Thus, there arise need to discover parameters which can measure a researcher’s contribution to his field. The most significant metric to measure the impact of a scientific work is citations. The Citation Indexes are utilized as scientometric tool to measure the research output of authors, articles and journals. This book chapter explores the existence of many such scientific parameters at both journal and author level. Further, authors make an earnest attempt to use them to measure the INTERNATIONALITY of peer-reviewed journals. They claim that already existing parameters alone are not sufficient for evaluation of internationality and explore new parameters for computing unbiased index both at journal and author level.
Keyword:- Impact Factor, H-Index, Snip, NLIQ, Citation Network, Cobb-Douglas Production Function.
1. INTRODUCTION
Research activity cannot be performed in isolation. It involves utilization of scientific ideas and techniques established by scholars in a given domain. Researcher experiment with ideas and publish work mostly in peer reviewed scientific journals. Though there are various motivations to cite previous work, authors cite to acknowledge and give credits to the contribution that were already done to the subject matter. This provides insight to build and maintain citation-based indexes and to endorse all Scientists and researchers who made major contributions in the advancement of research and development. In any respect, most scientific parameters have originated from Eugene Garfield’s concept of using bibliographic citations, Impact Factor being the most popular choice. While Impact Factor merely reflects number of citations per publication, H-index, another well known metric, incorporates both number of publications and number of citations in its influence computation. Although many prominent metrics are built using citation indexes, the research community often misuses these indexes and metrics by manipulating the citation count through practices like extensive self-citation, copious citation and at journal level, coercive citation. Researchers often suggest eliminating self-citations from total citation when computing influence. Arguably, self-citation cannot be labeled as unfair or unacceptable if an author choose to cite his related previous work. Copious citations is a practice in which multiple authors profusely cite each other with an intention to raise their h-index and similar influence-governing parameters. There have been similar cases of misuse, where Editor-in-Chief’s of two high impact factor journals are seen publishing extensively not only in their own journals but also into each others’. The journals continue to demonstrate such practice till date hampering the dignity and integrity of the structure of scholarly publications.
Another notable trend, followed opportunistically is when editors of low ranked journals persuade authors to cite their journal with an inclination to push their impact factors. Even though most authors view coercive citation as inappropriate, few low-ranked authors do not mind adding unrelated references and return the favor to publishing journals. If one journal coerces, to improve its rank, others gravitate and this becomes a trend that contaminates the whole publication process. One possible solution could be penalizing the usage of self-citations which would reduce, if does not dethrone, the coercive motivation.
Although journal’s prestige is reflected in its Impact factor, the overall framework of measuring “prestige” and for that matter its “internationality”, demands clear understanding and usage of deep rooted parameters. Scientific indicators like I.F., raw citations, h-index, h5-index, g-index, i-10 index can work as instruments to measure prestige; when it comes to measuring “internationality”, choosing a set of smart indicators and alternatively, defining new, unbiased ones become inevitable. There exist several vague definitions of the term “Internationality” which are based on including attributes like ISSN number, Editorial board members, reviewers, authors, Editors, Publishers and other associated entities of publishing community. Authors refute such definitions and claim that “internationality” should be defined as a quantifiable metric, which is devoid of any kind of misuse or manipulation, is unbiased towards origin of journal, author or article and calibrate journals on the basis of quality of research output it publishes.
The roadmap of the chapter is shown in Figure 1. The chapter is built on two pillars. The first cornerstone lay emphasis upon influence-measuring scientific parameters and examines in depth their merits and demerits. It defines ‘internationality’ of peer-reviewed journal and explores the capacity and degree to which these deep-rooted parameters contribute in computing ‘internationality’. A comparison of new parameter with the existing ones is explained and appropriateness in using these novel parameters is illustrated by citing examples and case study. The chapter moves forward to describe Cobb Douglas Model and mathematically proves its suitability for computing “internationality” of journals. Second pillar of the book chapter highlights the existing metric to measure scholastic influence and talk about new approaches toward author level metrics. It explores “influence” and “internationality” measuring metrics and moves ahead to explore the suitability of SIR and CISER model in their computation. In parallel, contribution of Citation Networks and Genealogy Tree in devising new indicators for author level metrics is also examined.
Figure 1 Road Map “Journal Metrics”
2. Background
The increasing number of journals instigates a need for quantification mechanism to evaluate the quality of the published research articles. The research carried out on such lines are based only on the usage of term “international”, without evaluating it without using appropriate measures or indices As a results, the literal definition of the word ‘international’is used in its minimal sense (Buela-Casal et al, 2006).
Impact Factor, devised by Eugene Garfield is the most widely used method to measure Journal’s influence. It’s a value calculated over time, and is based on the number of citations received in the previous two year divided by the number of articles published in same years. For a journal, all its citations of previous n years should be available to calculate impact factor for year (n+1) implying the computation of impact factor involves significant overhead. In spite of its drawback, the Journal Impact Factor is a popular method to measure the scientific influence of journals. Most of its variants too require the citation data of the preceding few years from all the indexed journals.
The other popular measure is the Elsevier’s Scopus which has a large collection of peer reviewed scholarly journals across various domains. Scopus utilizes its database to provide another type of journal metric to rank journals via SCImago Journal and Country Rank (SJR) portal. The SJR rank is a score evaluated by Scopus from the past five years’ data addressing small number of journals. It is claimed that SCI, Thomson Reuters are little more selective than Scopus.
A lot of research is carried by researchers across the globe. Ali Uzun (Ali, 2004) surveyed an American journal and a few European journals and analyzed bibliographic data about foreign authors and their national affiliation from five years of publication. He has then used a simple linear regression analysis to show that 60% of variation in the proportion of foreign-authored articles in the set of five journals over the selected years could be explained by the percentage of foreign members on the editorial boards of the journals. PandelisPerakakis et.al (Buela-Casal et. al, 2006)( Perakakis et.al ,2006) in two of the published works has proposed neuro-fuzzy system as a way to represent the problem of internationality. The neuro-fuzzy systems are the fuzzy sets of the weighted criteria linked by fuzzy rules in a multi-layer perceptron, whose output defuzzification gives measure of Journal Internationality Index akin to the Impact Factor for citations.
Aviles, Frank Pancho, Ramirez, IvonneSaidé (Aviles, Pancho, Ramirez, Ivonne, 2015) evaluated the internationality of several high visibility Library and information science journals by studying the permeation of international participation through the exploration of seven indicators: (1) the editorial and advisory boards, (2) peer review boards, (3) their database circulation, (4) authorship, (5) peer review evaluation criteria, (6) internationality of citations, (7) and citation impact.
NeelamJangid, SnehanshuSaha, Siddhant Gupta, Mukunda Rao J (Jangid et. al, 2015)(Jangid et. al, 2014) introduced a new metric, Journal Influence Score (JIS), which is calculated by applying Principal Component Analysis (PCA) and multiple linear regression (MLR) on citation parameters, extracted and processed from various scholarly articles in different domains, to obtain a score that gauges a journals impact. The higher the score, more the journal is valued and accepted. Journals ranking results are compared with ranks of SJR, which internally uses Google’s PageRank algorithm to calculate ranks. The results showed minimal error and the model performed reasonably well.
The general drawback in the above proposed models is that there is no common measure for the influence of journals across various domains. Also, there is a need to scrutinize the practices like self-citation which is the easiest way to increase the citation quotient of journals.Source Normalized Impact per Paper (SNIP) a measure proposed by Henk F. Moed (Henk, 2010) is the ratio of the journals citation count per paper and the citation potential in its subject field. It allows direct comparison of journals in different subject domains which is based on citation relationships. SNIP is based on citations from peer-reviewed papers to other peer-reviewed papers.
Minor modifications were done over the above mentioned method by Ludo Waltman et al (Waltman et. al, 2013). To correct the differences in citation practices, the source normalised approach is used. The benefit of this approach being, it does not require the classification of subject fields where the boundaries of fields are defined explicitly. There are some arguments around the original SNIP indicators’ properties that may be considered counter-intuitive. For instance, it is possible that additional citation has a negative correlation with journals SNIP value. The revised SNIP indicator that they have introduced in this paper is defined to avoid these kinds of counter-intuitive behaviours.
Carl Bergstrom and Jewin West, in the year 2007, initiated a project called Eigenfactor Project [9] which traces the routes in which the knowledge has passed down. They have framed citation networks which cater to the difference in standards of citation and different timelines in which the citations could occur. Their work claims that the probability of an article getting cited within a short span of time is very high. Thus it could be misleading if its importance is calculated based on the short time frame. Thus, the Eigen Score is calculated based on the citations obtained over a five year period.
Another major problem is the open access publishers who charge publication fees to authors and offer no editorial services. Such services offer an open gateway for the seekers of easy publication.Walt Crawford (Crawford, 2014) in 2014 has proposed the following steps to avoid predatory journals.
Step 1: To make a pertinent decision whether “The International Journal of A” is a good target, one must look for it in the Directory of Open Access Journals (doaj.org). If the journal is not in the directory, look for another journal in a similar subject category.
Step 2: If the journal is in DOAJ, explore its site, its APC policy, quality of English used, its editorial board members- whether they are real people. Otherwise start from step 1.
Step 3: Check whether article title over the past issues makes sense within the journal’s scope or if any author show up repeatedly within the past few issues. If so, go to step 1 again.
However, it is observed that the method is more of a manual investigation and hence ungainly and elaborate. The above case studies clearly indicate that there is a need for a mechanism which offers to address the issue of internationality modelling and interface estimation of peer-reviewed journals in the fields of science and technology.
The concept of citation index, Impact Factor and SJR ranking provide limited respite to the above mentioned challenges of distinguishing and ranking legitimate publishers from the fake entities. This gives plenty of motivation and reason to work on proving a journal’s credibility and integrity as well as ascertaining the quality, impact and influence of the publications.
3. “Influence” as a metric - Journal Level
The evaluation of journals has been carried out extensively in the past century. A general interest in understanding the trends and patterns by assessing the journals has existed since the 1920’s. However, the interests have varied persistently with respect to the context and paradigms of time. Academic institutions have been using publication information to evaluate their faculties during appraisals. The information mainly includes the impact factor (or influence) of the publishing journal. Higher the impact factor, more credible the scholar and his research is considered. However, it is important to gauge the integrity of journals which are pulling authors to publish. Recently, there are instances of predatory publishing that gained attention as well as worldwide criticism. Some illegitimate publishing houses charge authors with hefty publishing fee as they publish articles without providing any editorial service or peer-reviewing of the submitted article. Evaluation of a journal’s influence thus, requires a close scrutiny from all perspectives.
3.1. Introduction to Scientific Parameters
This section presents some of the widely known scientific parameters that describe metrics used for the understanding of the calculation of Internationality as defined under differing contexts and assumptions. It is essential to understand that the parameters described in this section is not an exhaustive list, especially due to the emergence of several of these parameters which have evolved over the years to stay relevant in the digital age.
3.1.1. Impact Factor (IF):The impact factor is one of the most important of all metrics that have been devised for understanding the international influence of a journal. This metric was put together by Eugene Garfield, the man who actively spearheaded research in the then newly-found area of Scientometrics and Bibliometrics back in the 1970’s. Thomson Reuters acquired the Eugene Garfield founded Institute of Scientific Information and is generally accredited with the introduction of the widespread use of the Impact Factor as a metric in their annual publication of the Journal Citation Reports, listing the computed IF measures for popular journals that it followed.
Understanding the Impact Factor as a measure is simple. The IF can be defined as the arithmetic mean of citations received by an article published as part of a journal. The IF, as described by Eugene Garfield, considers the citations received over the span of three years from the current year.Hence,