October 12, 2018

Evaluation of contributions of aquaculture researchers: The need to change researcher incentives to reduce salami science in aquaculture

“The number of articles published per year should never be used, under any circumstances, as a criterion in tenure or promotion decisions, or to rank academic institutions” (Siegel & Baveye, 2010)

 

The continued growth of the world's population requires continued growth of the supply of aquaculture products. Research that leads to farm innovation is essential to increase productivity, reduce costs, and overcome key bottlenecks to successful management and growth in aquaculture production. The insights, understanding, and abilities of a researcher to render key problems into researchable hypotheses, design and conduct experiments with rigorous methodologies, and interpret their results are key to developing practical solutions and providing guidance that contributes to further growth and development of aquaculture enterprises. Thus, the degree of contribution by an aquaculture scientist toward technological advances should ideally be the basis for their evaluation and reward structure.

Unfortunately, the principal metric often used by department chairpersons, science officers, and administrators of research programs to evaluate research accomplishments is the number of research articles published annually. Such a reward structure creates strong incentives to focus on generating large numbers of individual articles rather than a smaller number of articles of greater substance and integration. As a result, the incentive of more and more researchers is to consciously divide results obtained from studies into as many fragments as they believe may be publishable. Articles based on fragments of information carved out of a larger study have been described as the “least publishable units,” and the process itself has been widely referred to as “salami science,” “bologna publication,” or “trivial publication” (Nature Materials, 2005; U.S. Office of Research Integrity, 2017; Siegel & Baveye, 2010). Moreover, there are many examples of researchers who engage actively in salami science being rewarded by receiving more funding and promotions to higher rank than more ethical scientists who focus on quality research and publishing, clearly discouraging the latter. The problem of salami science is a global problem and is on the rise (U.S. Office of Research Integrity, 2017). There are increasing numbers of calls in many disciplines for a reorientation toward an emphasis on the core reason for conducting research, to “create and disseminate relevant knowledge that matters to society” (Arlinghaus, 2014). 

Publication of salami science–style fragments of data from well‐designed and coherent studies confound contributions to the literature and represent a breach of trust among the scientific community and the commitment to stakeholders. Without the context of the overall data set obtained from carefully developed hypotheses and subsequent testing, data fragments crafted into a stand‐alone article can be dangerously misleading. First, the value of the work may be distorted by artificially increasing the number of publications on the topic (U.S. Office of Research Integrity, 2017). Second, slicing studies into small fragments can imply that data presented are derived from a different subject sample with different hypotheses and give the impression that results have been repeated in other studies. Third, salami‐science articles can result in contradictory recommendations for aquaculture businesses. For example, an article that presents blood biochemistry data resulting from the use of a more expensive feed ingredient may show evidence of improved immunological status; however, a separate article published from the same study shows that the fish did not grow well in that treatment, and survival following a disease challenge was not improved. Thus, one article suggests that aquaculture producers should adopt the new, more expensive feed ingredient, while data from the same study presented in another publication show that the ingredient is unlikely to present benefits to aquaculture production. Accordingly, salami publications impede the progress of science and the development of aquaculture and should not be condoned. The U.S. Office of Research Integrity (2017) states clearly that “[r]esearchers therefore should avoid trivial or salami publication.”

Of equal concern is that the emphasis on numbers of publications leads researchers to think in terms of creating research designs that simply and quickly generate streams of data derived from a particular method, model, and so on, which can be incorporated into a series of related articles, thereby surrendering the objective of solving the most critical and pressing problems of relevant stakeholders. Examples of this abound, such as repeating tests of a particular ingredient for diets or immunological effects of a particular compound on one more species, when multiple studies have already shown clear effects in many related species.

The overemphasis on quantity over quality of the content of publications in the reward structure for researchers has also led, unfortunately, to a series of unfavorable realities in the world of scientific publishing. One of these is the proliferation of “courtesy” or “guest” authorship, which is an ethical violation. Authorship should include all those who have made a substantial contribution to the conception, design, implementation, and interpretation of the research, but other names should not be included as authors. Second, the emphasis on numbers of publications has led to a proliferation of journals, many of questionable practices. Third, the increase in the number of salami‐science manuscripts has led to a dramatic increase in the need for expert peer reviews. Reviewer fatigue has become a major challenge for top scientific journals as they seek reviews from top experts in the field who are besieged with requests to review articles in their field. In addition to increased demand for reviewers, the emphasis on slicing and dicing data has led to increasing problems of self‐plagiarism as authors cut and paste text from other manuscripts developed with other fragments of data derived from the same study. Self‐plagiarism, such as recycling text from other publications, has been characterized as “intellectual laziness” (Roig, 2010) but also violates the terms of copyright agreements signed by the author (Šupak‐Smolčić & Bilić‐Zulle, 2013). Finally, efforts to publish multiple articles from the same data set can lead to problems of duplicate publication. Researchers who merely repackage data in a different way from what was published in a different journal to increase their numbers for the year are engaging in activities that are the antithesis of responsible conduct of science. Those researchers are guilty of research misconduct with serious consequences when journal editors, reviewers, and supervisors discover their transgression.

This editorial is a call to administrators of university, science officers, and national and international research programs; promotion and tenure committees; reviewers of proposals; and those serving on award committees to seriously reexamine their evaluation criteria and policies explicitly from the perspective of whether those policies promote salami science, even if unintentionally. Respected scientific associations, such as the American Academy of Science, the National Association of Land‐Grant Universities, the World Aquaculture Society, and the American Fisheries Society, are also called upon to develop guidelines for the evaluation of researchers that provide greater emphasis on the quality of a researcher's contribution to science, to their disciplines, and, in the case of aquaculture, to its growth and development. The reward structure for researchers needs to be overhauled in many instances to focus on the quality of an individual's contributions to knowledge in their discipline and to the innovations and societal improvements that result from their work. There is a growing literature on potential mechanisms to restructure evaluation criteria based more on the researcher's “influence” (Donaldson & Cooke, 2013) and to refocus on quality as opposed to quantity by limiting the number of articles submitted to promotion and tenure committees. Donaldson and Cooke (2013) define the influence of a researcher's contribution as the “capacity to produce an effect on the advancement of scientific knowledge.” Given that science advances as new knowledge in a discipline accumulates, research contributions should be based on evidence of the application of research results to the development of new or more efficient aquaculture production technologies, the commercialization of new products, new policies, or other contributions to businesses and to society.

The challenge is to establish a more sophisticated means to evaluate the contribution of a researcher's efforts than relying only on the number of publications (Nature Materials, 2005). Various alternative proposals have been made to suggest alternative ways to evaluate the contributions of research scientists. Several indices (h‐index, g‐index, i10‐index) have been suggested, but these are still based on the number of articles.

Other approaches could involve evaluation rubrics that assign a weight to a score based on the number of articles published as one criterion to be added to the weighted score of a second criterion based on influence, impact, or contribution to the growth and development of aquaculture. The researcher would have the responsibility to provide evidence that their research results were in use by stakeholders and with some indication of the degree of its use. For example, letters from stakeholders could indicate whether the farm was actively using their results, to what purpose, and to what extent. Other approaches, such as those used by the U.S. Department of Agriculture‐Agricultural Research Service, assign points to categories such as the scope of the research, with greater points assigned for broader scopes that address complex questions or that require sophisticated research techniques, whether the studies answer important questions in the field; whether it results in contributions that add to scientific and professional knowledge; leads to important changes in existing products, methods, techniques, and processes; or opens significant new avenues for further study.

The Journal of the World Aquaculture Society (JWAS) is committed to its goal of contributing to the growth and development of aquaculture worldwide. JWAS strongly believes in this concept. It publishes articles that make a substantive contribution to the growth and development of aquaculture and rejects least‐publishable‐unit manuscripts and other manifestations of salami science. Research administrators are encouraged to reward their researchers with greater value assigned to publications in journals such as JWAS that manifest the quality of content and associated relevance to aquaculture science and the growth of aquaculture worldwide. 

REFERENCES
Share this:
Tags:

About Carole R. Engle

Former JWAS Editor in Chief - Carole Engle has devoted more than 35 years to aquaculture research, extension, and teaching. In addition to publishing more than 110 scientific articles, four books, and more than 80 extension publications, she has a combined 49 years of editorial experience, including serving as Editor-in-Chief of Aquaculture Economics and Management. She has been honored with numerous awards from the U.S. Aquaculture Society, the National Aquaculture Association, the Catfish Farmers of America, and the Catfish Farmers of Arkansas. Engle was on the faculty of the University of Arkansas at Pine Bluff (UAPB) for more than 27 years and, as Director of the Aquaculture/Fisheries Center and Chairperson of the Department of Aquaculture and Fisheries, led it through a period of rapid growth, development, and expansion.

Gold Sponsors

Magazine Articles

  • 2023

  • 2022

  • 2021

  • 2020

  • 2019

  • 2018

  • 2017

  • 2016