Journal List > J Korean Med Sci > v.30(11) > 1022768

TOOLS
Gasparyan, Yessirkepov, Voronov, Gerasimov, Kostyukova, and Kitas: Preserving the Integrity of Citations and References by All Stakeholders of Science Communication
Special Article
Editing, Writing & Publishing

Journal of Korean Medical Science 2015; 30(11): 1545-1552.

Published online: 16 October 2015

DOI: https://doi.org/10.3346/jkms.2015.30.11.1545

Preserving the Integrity of Citations and References by All Stakeholders of Science Communication

Armen Yuri Gasparyan1, Marlen Yessirkepov2, Alexander A. Voronov3, Alexey N. Gerasimov4, Elena I. Kostyukova5, George D. Kitas1, 6

1Departments of Rheumatology and Research and Development, Dudley Group NHS Foundation Trust (Teaching Trust of the University of Birmingham, UK), Russells Hall Hospital, Dudley, West Midlands, UK.

2Department of Biochemistry, Biology and Microbiology, South Kazakhstan State Pharmaceutical Academy, Shymkent, Kazakhstan.

3Department of Marketing and Trade Deals, Kuban State University, Krasnodar, Russian Federation.

4Department of Statistics and Econometrics, Stavropol State Agrarian University, Stavropol, Russian Federation.

5Faculty of Accounting and Finance, Department of Accounting Management Accounting, Stavropol State Agrarian University, Stavropol, Russian Federation.

6Arthritis Research UK Epidemiology Unit, University of Manchester, Manchester, UK.

Address for Correspondence: Armen Yuri Gasparyan, MD. Departments of Rheumatology and Research & Development, Dudley Group NHS Foundation Trust (Teaching Trust of the University of Birmingham, UK), Russells Hall Hospital, Dudley DY1 2HQ, West Midlands, UK. Tel: +44.1384-244842, Fax: +44.1384-244808, a.gasparyan@gmail.com

Received 1 September 2015       Accepted 8 September 2015

© 2015 The Korean Academy of Medical Sciences.

(open-access):

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Citations to scholarly items are building bricks for multidisciplinary science communication. Citation analyses are currently influencing individual career advancement and ranking of academic and research institutions worldwide. This article overviews the involvement of scientific authors, reviewers, editors, publishers, indexers, and learned associations in the citing and referencing to preserve the integrity of science communication. Authors are responsible for thorough bibliographic searches to select relevant references for their articles, comprehend main points, and cite them in an ethical way. Reviewers and editors may perform additional searches and recommend missing essential references. Publishers, in turn, are in a position to instruct their authors over the citations and references, provide tools for validation of references, and open access to bibliographies. Publicly available reference lists bear important information about the novelty and relatedness of the scholarly items with the published literature. Few editorial associations have dealt with the issue of citations and properly managed references. As a prime example, the International Committee of Medical Journal Editors (ICMJE) issued in December 2014 an updated set of recommendations on the need for citing primary literature and avoiding unethical references, which are applicable to the global scientific community. With the exponential growth of literature and related references, it is critically important to define functions of all stakeholders of science communication in curbing the issue of irrational and unethical citations and thereby improve the quality and indexability of scholarly journals.

Keywords: Science Communication, Bibliography as Topic, Periodicals as Topic, Citations, Publication Ethics

THE GLOBAL IMPORTANCE OF CITATIONS

Scientific citations are often viewed as tools for guiding readers across myriads of published sources, distinguishing innovations and preserving the integrity of bibliographic records (12). All contributors of scholarly articles are currently encouraged to upgrade their skills in citing, analyzing relevance, and managing references to ensure the accuracy of citations and completeness of references lists (3).
Technically correct and thoroughly validated citations add to the quality of reference lists and allow stakeholders of science communication to judge manuscripts and published articles fairly. As a good example, journal editors often pick potential reviewers from the reference lists, while reviewers scan the timelines of the references to judge the novelty and scope of the manuscripts(45).
In the era of 'big science' and expansion of online bibliographic databases, the implications of retrieving relevant sources and mapping citations are critical for interlinking large amounts of scholarly items, finding influential (highly cited) articles, and promoting periodicals, individual authors and academic institutions (Fig. 1). Tracking related sources and citations helps information facilitators build up networks of likeminded scholars and arrange professional interactions on online platforms such as ResearchGate® (6).
One of the basic principles of scholarly writing stands on distinguishing one's own ideas, words, and graphics from those adopted from published sources. Skilled authors always credit each adopted statement, known scientific fact and methods by citing related publications, giving preference to sources visible in reputed databases, libraries, or archiving platforms. Providing access to primary sources for reading, comprehending main points, and citing them in the proper context is the accepted norm for building up scholarly information (7). The availability of subscription or open-access channels ensures that authors are offered the basic opportunity for comprehending the context of cited sources and correctly organizing their references (3). Reviewers and editors with access to the currently available hubs of scholarly information are capable of suggesting pertinent changes and improving the quality and readability of the manuscripts by analyzing the relevance, ethical and technical correctness of bibliographies (89).
Prestigious citation-tracking databases, such as Scopus and Web of Science, rely heavily on the correctness of reference lists in the indexed items, which, in turn, influence the functionality of the bibliographic records and links between peer-reviewed sources (10). Indexers of these databases bear their share of responsibility, which may encompass regularly analyzing the correctness of references and delisting periodicals with massive technical mistakes and unethical referencing. By accepting for or continuing coverage of periodicals with poorly edited reference lists, indexers skew the citation-based impact indicators and distort researchers' and their institutions' profiles. The implications of such mistakes are far-reaching since current global ranking systems such as the QS World University Rankings, the Academic Ranking of World University and the Times Higher Education World University Rankings are all anchored on Scopus, the largest database of peer-reviewed journals, and powered by related reference and citation tracking (11).
Moreover, bibliographies have become a subject of semantic analyses over the past few decades, helping readers, researchers, and information scientists systematize the relatedness of scientific facts and explore new directions for research (121314).
The US National Library of Medicine recognized the growing importance of research in the field by introducing "bibliography as topic" to the Medical Subject Headings (MeSH) thesaurus in 2008. As a major term it has extensive links to other terms in the MeSH hierarchy, such as "documentation", "information science", "publications", and "bibliometrics". The term has been tagged on 15,620 items indexed by PubMed, with the highest number of 823 items published in 2013 (as of August 18, 2015).
Considering the growing importance of relevant and correct reference lists of scholarly items, we aimed to analyze the role of stakeholders of science communication in preserving the integrity of citations and references.

AUTHORS' PERSPECTIVE

Scientific authors are primary users of published items and ultimately responsible for selecting relevant and ethical citations (Table 1). They are engaged in science communication from the stage of retrieving scholarly items, publishing their own research, and adding new records to bibliographic databases. Although ideal citation practices are not defined, authors are advised to avoid excessive citations to their own works or other forms of manipulation, read full-texts of papers and cite them in the proper context, and give credit when credit is due (15). The authors, who strictly follow the instructions and adhere to the referencing style of the target journals, increase their chances of getting published (16).
Authors should realize that bibliographic databases rely heavily on the quantity and quality of the reference lists. The sustainability of citation-based databases depends on technical correctness and relevance of these lists. Despite the growth of databases and the availability of advanced search engines, several analyses across scientific disciplines have identified that 20%-25% of references cited by authors are erroneous and do not support their quotes or other statements (1718192021).
The implications of online bibliographic searches on the relevance and technical correctness of references have not been fully explored. However, it is likely that improved access to quality sources and familiarity with abstracts, keywords, and main points in the primary literature may improve the referencing and shape the whole publishing landscape (72122). There is evidence suggesting that training on bibliographic searches, librarian-guided workshops, and peer collaboration at the stage of undergraduate medical education improve the ongoing authors' referencing and essay writing skills in an ongoing fashion (23). Although the role of referencing courses in many other fields is unclear, it is likely that advanced searches through multidisciplinary citation databases, such as Scopus and Web of Science, will increase chances of retrieving highly relevant primary sources, which, in turn, will add to the overall quality of multidisciplinary journals.
The analysis of reference lists and records of citations on the databases may help authors deepen their searches for evidence-based and highly-cited items, which are usually endorsed by the wider scientific community. Highly-cited items (>100 times) are often methodological articles or those representing 'hot' topics and high level of evidence (e.g., large studies, trials, systematic reviews) (24252627).
Some, even highly-cited sources may contain erroneous, flawed, or otherwise unethical information, which is linked to the correction or retraction notices. The number of retracted publications is constantly growing while citations validating these items are still accumulating and influencing the scientific discourse (2829). In a landmark analysis of 5,000 citations to retracted biomedical articles, 93% of these citations were research-related, supporting the validity of predominantly fraudulent publications (30). Unfortunately, the majority of authors, who cite primary sources, overlook the related retraction notices and continue citing flawed and unethical items, thus adding inconsistency to the whole system of citation analyses (31).
The information explosion and expansion of interdisciplinary and cross-country research set common standards for writing, referencing, and indexing. Such standards, however, overlook the differences in the authors' citation behavior, which is compounded by the access to relevant bibliographic databases, professional, geographic, and language backgrounds (32333435). While citations have traditionally been viewed as credits to generators of new knowledge, related publications, or background reading, citing motives have diversified enormously in the past decades, and partly because of nonscientific reasons (36). For example, an analysis of 3,813 references from articles published in 2011 by three Brazilian orthopedics journals found that roughly 8% of them were to local sources and 41% of the analyzed articles did not contain any Brazilian reference (37). On the other extreme, preferential citations to local/national sources are commonplace in other countries, which skew quantitative and qualitative citation analyses further (38). Paradoxical citation patterns were documented for Korean physicists and mechanical engineers, who preferentially cited domestic sources in the articles published in Korean journals and mainstream science sources - in the articles published in the U.S., U.K., Netherlands, and Germany (39).
We analyzed the quantitative growth of references over the past decade across several countries (Table 2) and scientific disciplines (Table 3) based on the SCImago Journal & Country Rank data. The results indicate the exponential increase of indexed items and related references in the era of open access (from 2002 onward). Interestingly, the average number of references per document (Refs/Doc) has increased in most countries, but it is still the lowest in some non-Anglophone countries (e.g., Russia). Among biomedical disciplines, immunology has one of the highest value of Refs/Doc for 2013 (40.3) while nursing - the lowest (25.1).

REVIEWERS' PERSPECTIVE

Reviewers are expected to assess the manuscripts' validity, adherence to research reporting guidelines, clarity and consistency of writing, and correctness of references. The latter includes the adherence to the journal's style and limits of referencing, as well as the novelty and relevance of the listed references (40).
Currently used editorial management systems such as Aries systems® and ScholarOne® are equipped with tools for online bibliographic searches, which help reviewers analyze novelty and completeness of reference lists, overlaps with published bibliographies, and pick additional references to support their own statements. The editorial management systems refer the reviewers to the reference validation tools linked to CrossRef and PubMed.
Recommending additional references to support new ideas or debatable statements are helpful for substantiating the writing and avoiding plagiarism (41). Reviewers' analyses of the sequence and semantic links between in-text citations may reveal forms of plagiarisms, which are otherwise undetectable (42).
Reviewers may track statements and general knowledge, linked to irrational and multiple references, which artificially inflate the journals' and individuals' impact profiles. Suggesting replacements of irrelevant, old, nonpeer-reviewed, and secondary sources with more appropriate, evidence-based, and widely-visible ones are generally accepted and encouraged. Furthermore, reviewers may also perform searches through online databases and recommend relevant replacements for retracted, illegitimate (predatory), hardly accessible, secondary, or tertiary sources.
Attention should be paid to the verification of Web sites as references, which may change their contents and become inaccessible over the time. An analysis of reference lists of 2,822 articles, published in 2006-2013, revealed that mainstream general medical journals such as The Lancet and BMJ contain significantly less inaccessible Web references than their peripheral counterparts (43), pointing to more stringent verification practices in the formers.
Online items with permanent presence on the Internet, and particularly those with Digital Object Identifiers (DOIs), can be preferentially recommended for citation. Print and online items, including audio and video presentations, which do not pass through the traditional peer review, should be processed with caution since these may contain promotional, inappropriate, and potentially harmful information (4445).
Examples of tertiary sources, which are increasingly, but not always justifiably, cited in the indexed literature, are easily accessible encyclopedic articles from Wikipedia, and particularly those containing definitions and descriptions (46). Although Wikipedia pages are regularly edited, linked to a large number of primary references, and recommended as didactic resources for some disciplines (47), they may contain biased and inappropriately edited information (48). Other examples of tertiary and secondary references, which can be replaced by primary evidence-based sources, include monographs, textbooks, and dissertations (Table 1). These are abundantly cited by authors from nonmainstream science countries, encountering difficulties with accessing high-quality periodicals, which are accessible through the subscription paywalls.
An opportunity to recommend replacements and additions brought about an unethical practice of coercive citations. The practice involves both reviewers and editors, who coerce their authors to add citations to their articles and journals and make their decisions based on the willingness of the authors to cite additional references. An analysis of 6,672 responses in a multidisciplinary survey of authors demonstrated that 20% of the surveyees were coerced to cite and more than 40% were aware of that practice (49). The respondents reported 175 coercer journals in economics, sociology, psychology, and business.
Another recent analysis of 616 reviewer comments for the Journal of Psychosomatic Research found that one-third of recommended additional citations were to sources authored by the reviewers with 21% of these references turned irrelevant. Self-cites were twice more often in comments suggesting revision or acceptance than in those suggesting rejection (33% vs. 15%, P<0.001) (50). On the other hand, a report on 927 referee comments submitted to the Journal of the American Society of Information Science and Technology found little evidence that reviewers abuse the peer review system to play the 'citation games' (51), suggesting that there are differences in the journals' strategies for managing citations and references.

EDITORS' PERSPECTIVE

For decades, scholars educated at and affiliated to academic institutions of mainstream science countries have been considered as the best reviewers and editors with advanced skills of literature searches and referencing (52). Experts even advise to hire seasoned editors of reputable journals to curb the problem of inaccurate and unethical citations in nonmainstream science journals (53).
Some journal editors have raised concerns over the editorial biases, which are increasingly affecting the integrity of peer review, author-editor relationships, and authors' citation behavior (545556). Such biases have affected numerous disciplines and turned out to be related not to the professional credentials of the editors, but to their desire to attract 'easy' citations (57). The 'citation games' have damaged reputation of some low-impact journals and their desperate editors while the most reputable international journals and their editors distanced themselves from such unethical practices (5358).
Journal editors are in a position to detect and avoid irrelevant or coercive citations, and particularly those added during the peer review. By ensuring relevance of citations and proper credits to publicized facts and ideas, editors ensure the quality of their authors' writings (59). As a final resort, editors may monitor cited and citing sources post-publication to trace coercion or other manipulations in their journals and raise the issue of retracting items, which undermine the trustworthiness of citation networks (606162). Irrelevant or coercive citations are relatively new causes of retractions, which are often linked to more conspicuous causes such as fraud, plagiarism, authorship disputes, or honest errors (63).

PUBLISHERS' PERSPECTIVE

Publishers are capable of implementing strategies of proper citing and referencing by providing modes of citations to their own journal articles and upgrading their instructions for authors (64). Most journal instructions contain sections on technical accuracy, style (Vancouver, Harvard, or mixed systems), and limits of referencing. But it is increasingly important to highlight ethical principles of selecting and citing primary sources, which are missing in the majority of the instructions. As a precedent, an exemplary list of pointers to ethical citations is now available as a section of the editorial policies of BioMedCentral publisher (65).
An excellent initiative, which is aimed at improving the quality and reuse of reference lists, is set by Informa publishing company. The expert opinion and review article series of the publisher contain bibliographies where sources of considerable interest to readers are specifically marked. Elsevier and Springer went further and launched automatic alert services, informing their authors about citations to their articles appearing in other journals and tracked by CrossRef. Such an alert delves into the relatedness of cited and citing sources and facilitates networking of authors in the same and allied disciplines.
Enhancing visibility of bibliographies is yet another tool offered by publishers that may increase authors' responsibility over the reference lists and draw readers' attention to this section of scholarly articles. By appreciating the autonomous role of bibliographies in the distribution of scholarly information, large subscription publishers such as Springer, Elsevier and Emerald have opened access to their reference lists, leaving the rest of the subscription articles behind the paywalls. Such an initiative is supported further by the Open Citation Corpus, a repository of freely available citation data, which was recently launched to enhance the reuse of correctly recorded reference lists without copyright restrictions (66). The project started providing access to biomedical citations from PubMed Central, but its ultimate goal is to integrate with CrossRef and aggregators of reference lists such as CiteSeerX (citations in computer science; citeseerx.ist.psu.edu) and CitEc (Citations in Economics; citec.repec.org), cover most fields of science, and overcome limitations of databases and aggregators processing citation data (67). The Open Citation Corpus joined the Directory of Open Access Journals under the umbrella of the Infrastructure Services for Open Access in 2014 (68).

LEARNED ASSOCIATIONS' PERSPECTIVE

Only few learned associations have raised concerns about irrelevant or unethical citations, and formulated primary guidance in their recommendations and position papers. The most updated document containing points on the integrity of references is the "Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals" issued by the International Committee of Medical Journal Editors (ICMJE) in December, 2014 (69). The recommendations are of considerable interest to the multidisciplinary scientific community. They bring a balanced view on preferentially citing primary sources, keeping self-cites and bibliographies within justifiable limits, avoiding conference abstracts, "personal communications", and retracted items as references. Verifying technical correctness of each reference and adhering to the Vancouver style were also emphasized. The Council of Science Editors (CSE) included a section on citation manipulation, coercion and unethical boosting of the impact factors in the latest version of the "White Paper on Promoting Integrity in Scientific Journal Publications" (2012) (70). Finally, the American Society of Cells Biology released the widely endorsed San Francisco Declaration on Research Integrity (DORA, December 2012) that strongly encouraged citing primary scientific literature (71).

CONCLUSION

Bibliographies of scholarly articles reflect the overall quality and integrity of writing, editing, and publishing. With the exponential growth of scholarly articles and references across numerous disciplines, it is essential to adopt and enforce comprehensive strategies aimed at retrieving relevant references while writing, validating citations at the peer review, and checking their technical correctness at the copyediting and proofreading. Currently available digital tools for interlinking millions of scholarly items, particularly through the platform of CrossRef, help publishers manage the reference lists and avoid technical mistakes, which were common in the pre-digital era (7273).
One of the latest accounts of the inaccuracy rates (7.6% inaccurate quotations out of 3,840 scanned ones in ten orthopedic journals) suggest that the irrelevance of citations is still unacceptably high and is a much bigger issue than technical mistakes (74). The open access movement that facilitates easy access to both full-texts and bibliographies may improve the relevance of citing and draw attention of readers to the quality of reference lists. Selectively opening access to bibliographies by some subscription publishers is an interim measure that may add to the quality of bibliographies and form the basis for reference aggregation by specifically designed platforms.
Global and regional editorial associations fill the gaps in the quality assurance of citations and references by (re)drafting their recommendations and paying attention to the emerging issues of access, relevance, and ethics rather than technical mistakes. Defining responsibilities of all stakeholders of science communication in curbing these issues is an emerging task for editorial associations. Their recommendations and position statements are warranted for all scholarly journals, but it seems that small and nonmainstream science journals, struggling to get indexed, may benefit from the instructive recommendations more.
Finally, indexing services may play a decisive role in improving the quality of citations and references by tightening indexing criteria and discontinuing coverage of periodicals, which devalue the citation analyses. Global indexing services such as Scopus and Web of Science have traditionally prioritized the quantity, language, technical correctness, and visibility of references. The time has come to incorporate ethics and relevance of references in the expanding list of the indexing criteria.

Figures and Tables

Fig. 1

The expanding role of citations and references of scholarly articles.

jkms-30-1545-g001
Table 1

Examples of recommended and unacceptable references for scholarly articles

jkms-30-1545-i001
Recommended references Unacceptable references
Regular articles of peer-reviewed, indexed in global databases, and widely-visible periodicals with Digital Object Identifiers (DOIs) Papers in nonpeer-reviewed magazines, newspapers, and illegitimate (predatory) journals
Peer-reviewed and indexed in reputable databases articles - output of PhD dissertations and degree theses PhD dissertations, theses, annotations and other nonpeer-reviewed outputs of degree projects
Chapters of widely visible handbooks and monographs with DOIs Nondigitized, hardly visible for the global community and outdated handbooks, textbooks, and monographs
Web pages of reputable and permanently preserved online resources of professional information (blogs, listservs, discussion platforms, professional forums controlled by moderators) Web pages of uncontrolled, poorly edited, and otherwise unreliable online resources
Widely visible online and print guidelines of large professional associations and other types of grey literature Nonevidence-based and hardly visible recommendations of small societies, instructions and orders of local administrative organizations (ministries)
Video articles with DOIs and other attributes of scholarly articles Audio and video materials from uncontrolled, unchecked and poorly edited Web resources (e.g., promotional YouTube films containing controversial and potentially harmful information)
Retracted items
Table 2

Quantitative analysis of the growth of indexed items and related references across several countries over the period of 2002-2013

jkms-30-1545-i002
Country 2002 2013
Total journals Total docs Total refs Refs/doc Total journals Total docs Total refs Refs/doc
Netherlands 1,341 125,035 3,459,574 30.1 1,839 230,393 8,773,836 40.9
UK 3,646 285,197 7,812,578 29.7 5,411 531,491 19,203,587 39.8
USA 4,852 481,126 11,743,853 24.6 6,035 735,079 23,920,449 35.8
Germany 884 74,413 1,743,192 27.8 1,336 219,677 5,199,750 33.9
Greece 13 2,345 50,352 25.4 65 7,036 212,521 28.5
Italy 295 13,084 274,784 19.7 487 26,728 711,718 27.1
Croatia 64 1,999 38,409 18.3 135 4,918 132,708 26.8
Turkey 55 2,068 42,617 17.1 178 10,427 255,302 24.8
Iran 8 296 5,474 19.0 133 6,896 184,214 23.4
Korea 50 3,980 79,108 16.3 193 18,258 418,531 23.4
India 176 11,060 171,727 14.2 451 44,009 896,896 20.7
China 415 42,592 299,405 5.7 567 113,433 2,067,933 18.5
Japan 460 34,851 586,172 16.1 503 40,728 765,815 17.9
Russian Federation 264 36,978 368,851 11.3 219 22,443 380,465 17.9

The findings were obtained from the SCImago Journal & Country Rank platform on August 26, 2015 (http://www.scimagojr.com/journalrank.php). Ranking of countries is based on values of references per document (refs/doc) in 2013.

Table 3

Quantitative analysis of the growth of indexed items and related references across several scientific disciplines over the period of 2002-2013

jkms-30-1545-i003
Subject category 2002 2013
Total journals Total docs Total refs Refs/doc Total journals Total docs Total refs Refs/doc
Arts & Humanities 305 14,086 448,893 34.1 427 26,742 991,318 41.2
Immunology and Allergy 130 13,011 423,843 36.4 178 20,589 791,704 40.3
Communication 70 1,867 60,533 31.6 232 7,233 263,049 38.5
Language and Linguistics 163 4,592 157,905 32.6 560 14,872 574,266 38.0
Social sciences 273 5,647 186,004 23.6 393 16,390 643,494 37.7
Chemistry 313 48,241 1,264,237 27.3 383 93,163 3,478,400 34.3
Economics, Econometrics & Finance 62 1,333 39,116 25.1 231 9,513 281,823 33.6
Rheumatology 39 4,272 115,026 25.9 57 7,428 221,727 31.6
Pharmaceutical science 108 12,613 222,772 20.6 201 22,231 712,884 28.7
Library and information sciences 101 3,787 65,237 22.5 196 8,627 237,102 28.0
Orthopedics and Sports medicine 134 11,751 273,578 23.3 218 22,825 637,268 27.5
Cardiology and Cardiovascular medicine 223 26,686 625,571 22.6 334 45,546 1,147,864 26.6
Nursing 82 5,718 82,363 17.7 112 9,991 220,227 25.1
Mathematics 198 11,826 206,085 18.6 351 25,285 559,118 24.0

The findings were obtained from the SCImago Journal & Country Rank platform on August 26, 2015 (http://www.scimagojr.com/journalrank.php). Ranking of disciplines is based on values of references per document (refs/doc) in 2013.

Notes

DISCLOSURE The authors have no conflicts of interest to disclose.

AUTHOR CONTRIBUTION All authors contributed to the study design, interpretation of the literature data, and the manuscript drafting. All authors read and approved the final version of the manuscript for publication.

References

1. Biebuyck JF. Concerning the ethics and accuracy of scientific citations. Anesthesiology. 1992; 77:1–2.
2. On citing well. Nat Chem Bio. 2010; 6:79.
3. Bryson D. Using research papers: citations, referencing and plagiarism. J Vis Commun Med. 2012; 35:82–84.
4. Annesley TM. Giving credit: citations and references. Clin Chem. 2011; 57:14–17.
5. Gasparyan AY, Kitas GD. Best peer reviewers and the quality of peer review in biomedical journals. Croat Med J. 2012; 53:386–389.
6. Review of ResearchGate: Pros and cons and recommendations. accessed on 26 August 2015. Available at http://blogs.exeter.ac.uk/openresearchexeter/2013/11/06/74/.
7. Gasparyan AY, Ayvazyan L, Blackmore H, Kitas GD. Writing a narrative biomedical review: considerations for authors, peer reviewers, and editors. Rheumatol Int. 2011; 31:1409–1417.
8. Ezeala C, Nweke I, Ezeala M. Common errors in manuscripts submitted to medical science journals. Ann Med Health Sci Res. 2013; 3:376–379.
9. Barroga EF. Reference accuracy: authors', reviewers', editors', and publishers' contributions. J Korean Med Sci. 2014; 29:1587–1589.
10. Gasparyan AY, Ayvazyan L, Kitas GD. Multidisciplinary bibliographic databases. J Korean Med Sci. 2013; 28:1270–1275.
11. Hassan SU, Muazzam A. Analyzing institutional research publications, citations and collaboration activities in sustainable development and its sub-areas, with particular reference to Asian institutions. J Sustain Sci Manag. 2013; 8:180–190.
12. Hjørland B. Citation analysis: a social and dynamic approach to knowledge organization. Inf Process Manag. 2013; 49:1313–1325.
13. Chen J, Zhuge H. Summarization of scientific documents by detecting common facts in citations. Future Gener Comput Syst. 2014; 32:246–252.
14. Hristovski D, Dinevski D, Kastrin A, Rindflesch TC. Biomedical question answering using semantic relations. BMC Bioinformatics. 2015; 16:6.
15. Todd PA, Ladle RJ. Citations: poor practices by authors reduce their value. Nature. 2008; 451:244.
16. Hong ST. Ten tips for authors of scientific articles. J Korean Med Sci. 2014; 29:1035–1037.
17. Davids JR, Weigl DM, Edmonds JP, Blackhurst DW. Reference accuracy in peer-reviewed pediatric orthopaedic literature. J Bone Joint Surg Am. 2010; 92:1155–1161.
18. Haussmann NS, McIntyre T, Bumby AJ, Loubser MJ. Referencing practices in physical geography: how well do we cite what we write? Prog Phys Geogr. 2013; 0309133313482135.
19. Todd PA, Guest JR, Lu J, Chou LM. One in four citations in marine biology papers is inappropriate. Mar Ecol Prog Ser. 2010; 408:299–303.
20. Teixeira MC, Thomaz SM, Michelan TS, Mormul RP, Meurer T, Fasolli JV, Silveira MJ. Incorrect citations give unfair credit to review authors in ecology journals. PLoS One. 2013; 8:e81871.
21. Luo M, Li CC, Molina D 4th, Andersen CR, Panchbhavi VK. Accuracy of citation and quotation in foot and ankle surgery journals. Foot Ankle Int. 2013; 34:949–955.
22. Kangas A, Hujala T. Challenges in publishing: producing, assuring, and communicating quality. Silva Fenn. 2015; 49:1304.
23. Krasne S, Stevens CD, Wilkerson L. Improving medical literature sourcing by first-year medical students in problem-based learning: outcomes of early interventions. Acad Med. 2014; 89:1069–1074.
24. Barbosa FG, Schneck F. Characteristics of the top-cited papers in species distribution predictive models. Ecol Modell. 2015; 313:77–83.
25. Yang X, Gu J, Yan H, Xu Z, Ren B, Yang Y, Yang X, Chen Q, Tan S. The highly-cited Electrocardiogram-related articles in science citation index expanded: characteristics and hotspots. J Electrocardiol. 2014; 47:738–744.
26. Eshraghi A, Osman NA, Gholizadeh H, Ali S, Shadgan B. 100 top-cited scientific papers in limb prosthetics. Biomed Eng Online. 2013; 12:119.
27. Tao T, Zhao X, Lou J, Bo L, Wang F, Li J, Deng X. The top cited clinical research articles on sepsis: a bibliometric analysis. Crit Care. 2012; 16:R110.
28. Marcus A, Oransky I. What studies of retractions tell us. J Microbiol Biol Educ. 2014; 15:151–154.
29. Gasparyan AY, Ayvazyan L, Akazhanov NA, Kitas GD. Self-correction in biomedical publications and the scientific impact. Croat Med J. 2014; 55:61–72.
30. Steen RG. Retractions in the medical literature: how many patients are put at risk by flawed research? J Med Ethics. 2011; 37:688–692.
31. Neale AV, Dailey RK, Abrams J. Analysis of citations to biomedical articles affected by scientific misconduct. Sci Eng Ethics. 2010; 16:251–261.
32. Adair JG, Vohra N. The explosion of knowledge, references, and citations. Psychology's unique response to a crisis. Am Psychol. 2003; 58:15–23.
33. Rasmussen K, Jørgensen KJ, Gøtzsche PC. Citations of scientific results and conflicts of interest: the case of mammography screening. Evid Based Med. 2013; 18:83–89.
34. Willett P. Readers' perceptions of authors' citation behaviour. J Doc. 2013; 69:145–156.
35. Hu G, Wang G. Disciplinary and ethnolinguistic influences on citation in research articles. J Engl Acad Purp. 2014; 14:14–28.
36. Bornmann L, Daniel HD. What do citation counts measure? A review of studies on citing behavior. J Doc. 2008; 64:45–80.
37. Teixeira RK, Yamaki VN, Rosa Rde C, de Barros RS, Botelho NM. Dominance of foreign citations in Brazilian orthopedics journals. Rev Bras Ortop. 2014; 49:668–670.
38. Erikson MG, Erlandson P. A taxonomy of motives to cite. Soc Stud Sci. 2014; 44:625–637.
39. Kim MJ. Citation patterns of Korean physicists and mechanical engineers: differences by type of publication source and type of authorship. Scientometrics. 2002; 55:421–436.
40. Azer SA, Ramani S, Peterson R. Becoming a peer reviewer to medical education journals. Med Teach. 2012; 34:698–704.
41. Roig M. Avoiding unethical writing practices. Food Chem Toxicol. 2012; 50:3385–3387.
42. Gipp B, Meuschke N, Breitinger C. Citation-based plagiarism detection: Practicability on a large-scale scientific corpus. J Assoc Inf Sci Technol. 2014; 65:1527–1540.
43. Habibzadeh P. Decay of references to Web sites in articles published in general medical journals: mainstream vs small journals. Appl Clin Inform. 2013; 4:455–464.
44. Butler DP, Perry F, Shah Z, Leon-Villapalos J. The quality of video information on burn first aid available on YouTube. Burns. 2013; 39:856–859.
45. Frohlich DO, Birnbrauer K. Discrepancies in health information found on web sites discussing cures for inflammatory bowel disease, an "incurable" disease. Inflamm Bowel Dis. 2014; 20:458–463.
46. Bould MD, Hladkowicz ES, Pigford AA, Ufholz LA, Postonogova T, Shin E, Boet S. References that anyone can edit: review of Wikipedia citations in peer reviewed health science literature. BMJ. 2014; 348:g1585.
47. Kräenbring J, Monzon Penza T, Gutmann J, Muehlich S, Zolk O, Wojnowski L, Maas R, Engelhardt S, Sarikas A. Accuracy and completeness of drug information in Wikipedia: a comparison with standard textbooks of pharmacology. PLoS One. 2014; 9:e106930.
48. Heilman JM, West AG. Wikipedia and medicine: quantifying readership, editors, and the significance of natural language. J Med Internet Res. 2015; 17:e62.
49. Wilhite AW, Fong EA. Scientific publications. Coercive citation in academic publishing. Science. 2012; 335:542–543.
50. Thombs BD, Levis AW, Razykov I, Syamchandra A, Leentjens AF, Levenson JL, Lumley MA. Potentially coercive self-citation by peer reviewers: a cross-sectional study. J Psychosom Res. 2015; 78:1–6.
51. Sugimoto CR, Cronin B. Citation gamesmanship: testing for evidence of ego bias in peer review. Scientometrics. 2013; 95:851–862.
52. Gasparyan AY. Selecting your editorial board: maintaining standards. J Korean Med Sci. 2013; 28:972–973.
53. Loyola RD, Diniz Filho JA. Citations: ethical ways to grow impact. Nature. 2013; 501:492.
54. Falagas ME, Alexiou VG. Editors may inappropriately influence authors' decisions regarding selection of references in scientific articles. Int J Impot Res. 2007; 19:443–445.
55. Krell FT. Should editors influence journal impact factors? Learn Publ. 2010; 23:59–62.
56. García JA, Rodriguez-Sánchez R, Fdez-Valdivia J. The author-editor game. Scientometrics. 2015; 104:361–380.
57. Lăzăroiu G. On Citation Ethics: editorial shenanigans to boost impact factor. Contemp Readings Law Soc Justice. 2013; 82–87.
58. Huggett S. Journal bibliometrics indicators and citation ethics: a discussion of current issues. Atherosclerosis. 2013; 230:275–277.
59. Romano NC Jr. Journal self-citation V: coercive journal self-citation - manipulations to increase impact factors may do more harm than good in the long run. Comm Assoc Inf Syst. 2009; 25:5.
60. Yu T, Yu G, Wang MY. Classification method for detecting coercive self-citation in journals. J Informetr. 2014; 8:123–135.
61. Citation manipulation: journal retracts paper because author boosted references to a journal he edits. accessed on 26 August 2015. Available at http://retractionwatch.com/2014/02/03/citation-manipulation-journal-retracts-paper-because-author-boosted-references-to-a-journal-he-edits/.
62. Journal retracts two papers after being caught manipulating citations. accessed on 26 August 2015. Available at http://retractionwatch.com/2013/08/30/journal-retracts-two-papers-after-being-caught-manipulating-citations/.
63. Decullier E, Huot L, Samson G, Maisonneuve H. Visibility of retractions: a cross-sectional one-year study. BMC Res Notes. 2013; 6:238.
64. Gasparyan AY, Ayvazyan L, Gorin SV, Kitas GD. Upgrading instructions for authors of scholarly journals. Croat Med J. 2014; 55:271–280.
65. Editorial policies. citations. accessed on 26 August 2015. Available at http://www.biomedcentral.com/about/editorialpolicies#Citations.
66. Shotton D. Publishing: open citations. Nature. 2013; 502:295–297.
67. Peroni S, Dutton A, Gray T, Shotton D. Setting our bibliographic references free: towards open citation data. J Doc. 2015; 71:253–277.
68. Open Citation Corpus. accessed on 26 August 2015. Available at http://is4oa.org/services/open-citations-corpus/.
70. CSE's White Paper on Promoting Integrity in Scientific Journal Publications, 2012 Update. accessed on 26 August 2015. Available at http://www.councilscienceeditors.org/wp-content/uploads/entire_whitepaper.pdf.
71. American Society for Cell Biology. San Francisco Declaration on Research Assessment. Putting science into the assessment of research. accessed on 26 August 2015. Available at http://www.ascb.org/dora/wp-content/uploads/2015/07/SFDeclarationFINAL.pdf.
72. Lee SY, Lee JS. A survey of reference accuracy in two Asian dermatologic journals (the Journal of Dermatology and the Korean Journal of Dermatology). Int J Dermatol. 1999; 38:357–360.
73. O'Connor AE, Lukin W, Eriksson L, O'Connor C. Improvement in the accuracy of references in the journal Emergency Medicine Australasia. Emerg Med Australas. 2013; 25:64–67.
74. Buijze GA, Weening AA, Poolman RW, Bhandari M, Ring D. Predictors of the accuracy of quotation of references in peer-reviewed orthopaedic literature in relation to publications on the scaphoid. J Bone Joint Surg Br. 2012; 94:276–280.
TOOLS
ORCID iDs

Armen Yuri Gasparyan
https://orcid.org/http://orcid.org/0000-0001-8749-6018

Marlen Yessirkepov
https://orcid.org/http://orcid.org/0000-0003-2511-6918

Alexander A. Voronov
https://orcid.org/http://orcid.org/0000-0001-8505-7345

Alexey N. Gerasimov
https://orcid.org/http://orcid.org/0000-0002-1244-4755

Elena I. Kostyukova
https://orcid.org/http://orcid.org/0000-0002-8393-6001

George D. Kitas
https://orcid.org/http://orcid.org/0000-0002-0828-6176

Similar articles