Peer Reviewers in Central Asia: Publons Based Analysis

Sakir Ahmed; Marlen Yessirkepov

doi:10.3346/jkms.2021.36.e169

Abstract

Background

The five Central Asian republics comprise of Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, and Uzbekistan. Their research and publication activities are gradually improving but there is limited data on how good their peer reviewing practices are.

Methods

We have use the Publons database to extract information on the reviewers registered including the number of verified review, Publons award winners, and top universities in the domain of peer reviewing. This has been analysed overall and country wise.

Results

Of 15,764 researchers registered on Publons, only 370 (11.7%) have verified records of peer-reviewing. There are 8 Publons award winners. There is great heterogeneity in the number of active reviewers across the five countries. Kazakhstan and Uzbekistan account for more than 90% of verified reviewers. Only Kazakhstan has more than 100 active reviewers and 6 Publons award recipients. Amongst the top 20 reviewers from Central Asia, half of them are from the Nazarbayev University, Nur-Sultan, Kazakhstan. Three countries have less than 10 universities registered on Publons.

Conclusion

Central Asia has a good number of peer reviewers on Publons though only a minority of researchers are involved in peer reviewing. However, the heterogeneity between the nations can be best dealt with by promoting awareness and international networking including e-learning and mentoring programs.

INTRODUCTION

Central Asia (CA) has a rich scientific temperament but has reduced visibility in the field of scientific literature. This may stem from the previous dependence on the Russian language and reduced exposure to international science.1 Things are slowly changing. Some countries are doing well and some have yet to catch up in terms of publications.2 Previous analyses have focussed on publications from these regions.2 3 4 The increasing number of manuscripts portrays growing scientific curiosity, research intent and the increasing ability to understand the intricacies and ethics of publication. Complementary to scientific publishing, another integral requirement for the development of scientific rationale and thinking is peer-reviewing.5

Peer reviewing is an integral part of the research publication process, and often it is considered a thankless job.6 However, mainly thanks to the Publons initiative, more and more researchers are being recognised for their contributions to peer review.7 The coronavirus disease 2019 pandemic has stressed the peer review process8 and might be the refining fire that forges the next generation of peer reviewers. A peer reviewer, who is below par, may disrupt the trust that underlies this privileged gate-keeping function with some offhand remark.9 Thus, it is pertinent to have peer reviewers at par with the best in the world.

Overall, Asia is estimated to have a good pool of upcoming reviewers.10 In the field of publishing, Kazakhstan seems to be doing well.11 A recent analysis of the top institutes in CA and the neighbouring region has shown dominance by China in various Publons ratings.4 The same Publons platform provides a unique opportunity to explore the reviewing experiences and capacities of these nations.7 Publons was initially launched to provide credit for reviewers. However, it has expanded laterally and now also includes editorial records. After being brought under Clarivate Analytics, it is now also laterally integrated with the Web of Science from which publication information is synchronized. The ‘Publons Reviewer Connect’ is an artificial intelligence based program that can help editors identify potential reviewers.12 It can also synchronize records with the ORCID database.13 Thus, it is one of the best available tools to analyse reviewer profiles, in relationship to the reviewers' publications and editorial activities. Thus, we have attempted to analyse the numbers and expertise of peer reviews as well as top universities involved in peer reviewing from various Central Asian countries using Publons.

METHODS

There are 5 Central Asian republics with diverse research evaluation and publication strategies: Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan. The inbuilt search engine of Publons was used to extract data on May the 10^th, 2021.

Data import

First, the total number of researchers registered for each of these countries was noted. Then, the total number of reviewers with at least one verified review, with more than one verified review, the number of Publons recognized Top Reviewers, and the number of reviewers with at least one verified review in the last 12 months was extracted (Table 1). Also, the total number of reviews for each country as well as the total number of reviews for each country in the preceding 12 months was totalled. Finally, the top institutes of CA having the most number of reviews to their name, and the top 10 institutes per country were imported from Publons. Additionally, the number of Web of Science publications for the reviewers and the top institutes were also noted.

Table 1

Publons resource used for estimation of different parameters for the analysis

Parameters	Uniform resource locator (URL)^a
Total number of researchers	https://publons.com/researcher/?country=181&order_by=num_reviews
Number with at least 1 verified review	https://publons.com/researcher/?country=181&order_by=num_reviews
Number with more than 1 verified review	https://publons.com/researcher/?country=181&order_by=num_reviews
Number with at least 1 verified review in the last 12 mon	https://publons.com/researcher/?country=181&is_last_twelve_months=1&order_by=num_reviews
Number who received Publons top reviewer recognition	https://publons.com/researcher/?country=181&prw=1&order_by=num_reviews
Total number of reviews per country	https://publons.com/researcher/?country=181&order_by=num_reviews
Total number of reviews per country in last 12 mon	https://publons.com/researcher/?country=181&is_last_twelve_months=1&order_by=num_reviews
Top institutes with maximum number of verified reviews	https://publons.com/researcher/?country=181&country=184&country=3&country=148&country=48&order_by=num_reviews
Top reviewers and their universities per country	https://publons.com/researcher/?country=181&order_by=num_reviews

^aCountry code is provided for Kazakhstan (“181”) as an example.

Data analysis

Normality of data was assessed by the Shapiro-Wilk test. Descriptive statistics are described along with graphical presentations including bar-graphs, pie-charts and scatter-plots as relevant.

Software

Data was formatted into spreadsheets in MS Excel and then imported into the R statistical environment version 4.0.3. Both Excel and R packages ggpubr, dplyr, GGally, ggplot2 were used for data analysis and data visualization.

RESULTS

The total number of researchers from CA registered on Publons was 15,764 and the countrywide distribution is mentioned in Table 2. However, only 370 (11.7%) have at least 1 verified review (Fig. 1). This varies amongst the countries (Fig. 2). Six reviewers from Kazakhstan and one each from Uzbekistan and Kyrgyzstan have achieved the distinction of being awarded the Publons Reviewer Award.

Table 2

Number of registered researcher per Central Asian country

Country	Total researchers	Number with verified reviews, No. (%)
Kazakhstan	11,480	250 (2.18)
Kyrgyzstan	1,608	25 (1.55)
Tajikistan	145	4 (2.76)
Turkmenistan	14	2 (14.29)
Uzbekistan	2,517	89 (3.54)

Fig. 1

Distribution of total researchers (A) versus researchers with at least 1 verified review (B).

Fig. 2

Country-wise distribution of number of reviewers (A) with at least 1 verified review within the last 12 months (B) with more than 1 verified review and (C) with at least 1 verified review.

The top 20 reviewers across CA with the maximum number of verified reviews are listed in Table 3. Ten of the top 20 reviewers belong to Nazarbayev University, Nur-Sultan, Kazakhstan. Seven of the top reviewers are also active editors with a different number of editorial records to their names.

Table 3

Top 20 reviewers with the maximum number of verified reviews on Publons along with their Universities

Central Asia reviewer rank	Researchers	University	No. of verified reviews
1	Timur Sh. Atabaev	Nazarbayev University	1,353
2	Abduzhappar Gaipov	Nazarbayev University	255
3	Ruslan Kalendar	University of Helsinki	232
4	Dr. Gabriel A. Ogunmola	[Not declared]	231
5	Ali Shafiei	Nazarbayev University	199
6	Bobomurat Ahmedov	Ulugh Beg Astronomical Institute (UBAI)	174
7	Riccardo Pelizzo	Nazarbayev University	171
8	Enrico Marsili	Nazarbayev University	167
9	Shavkat Ayupov	Institute of Mathematics Named After V.I. Romanovsky	142
10	Dilfuza Egamberdieva	National University of Uzbekistan	125
11	Maxim V Zdorovets	L.N. Gumilyov Eurasian National University	118
12	Dimitri Poddighe	Nazarbayev University School of Medicine	107
13	Bulat Kenessov	Al-Farabi Kazakh National University	105
14	Utkir A. Rozikov	Institute of Mathematics Named After V.I. Romanovsky	95
15	Stefanos Xenarios	Nazarbayev University	94
16	Luis Rojas-Solorzano	Nazarbayev University	79
17	Artem Kozlovskiy	Institute of Nuclear Physics, Almaty, Kazakhstan	79
18	Dalila B.M.M. Fontes	FEP Fac Economia Universidade do Porto	76
19	Mehdi Bagheri	Nazarbayev University	73
20	Vsevolod A. Peshkov	Nazarbayev University	73

The top 10 universities with the maximum number of reviews per country have been listed in Table 4. However, this list is abbreviated for 3 countries that have less than 10 universities registered on Publons. Fig. 3 shows the relationship between the number of verified reviews and publication from these 29 universities across the five countries.

Table 4

The top 10 Universities of each Central Asian republic as per number of verified reviews in Publons

Country	Publons rank (Peer review)	1	2	3	4	5	6	7	8	9	10
Kazakhstan	University	Nazarbayev University	Al-Farabi Kazakh National University	Institute of Nuclear Physics, Almaty, Kazakhstan	L.N. Gumilyov Eurasian National University	Nazarbayev University School of Medicine	KIMEP University	Satbayev University	Kazakh British Technical University	Kazakh National Medical University named after S.D. Asfendiyarov	Semey State Medical University
	Verified reviews	3,543	348	254	236	146	101	81	45	28	26
	Reviews in last 12 mon	872	94	174	164	70	23	26	12	4	7
	Publications	3,468	5,081	850	2,505	357	88	1,893	308	172	107
Kyrgyzstan	University	Kyrgyz Turkish Manas University	National University	Kyrgyz State Medical Academy	Osh State University	American University of Central Asia	Research Station of the Russian Academy of Sciences in Bishkek, Research Station RAS	Jusup Balasagyn Kyrgyz National University
	Verified reviews	29	23	12	10	9	5	1
	Reviews in last 12 mon	11	4	1	4	2	3	1
	Publications	278	75	477	54	16	160	31
Tajikistan	University	S.U. Umarov Physical-Technical Institute of Academy of Sciences of the Republic of Tajikistan	Academy of Sciences of Republic of Tajikistan
	Verified reviews	5	1
	Reviews in last 12 mon	2	-
	Publications	-	16
Turkmenistan	University	No registered University
	Verified reviews
	Reviews in last 12 mon
	Publications
Uzbekistan	University	Institute of Mathematics Named After V.I. Romanovsky	Ulugh Beg Astronomical Institute (UBAI)	National University of Uzbekistan	Yunusov Institute of Chemistry of Plant Substances	Tashkent Institute of Irrigation and Agricultural Mechanization Engineers	Republican Specialized Scientific-Practical Medical Center of Therapy and Medical Rehabilitation	Institute of Botany, Uzbekistan	Westminster International University in Tashkent	Institute of Ion-Plasma & Laser Technologies Named After U.A. Arifov	Tashkent Pediatric Medical Institute
	Verified reviews	304	215	133	29	26	23	22	12	10	9
	Reviews in last 12 mon	47	67	36	21	10	10	8	1	6	6
	Publications	1,510	926	762	373	89	201	102	25	774	41

Fig. 3

Scatterplot of number of publications versus total verified reviews for top 10 Universities from Central Asia with the lower panel zooming in on the X-axis between 0 to 350. Note that Turkmenistan has no universities registered on Publons while Tajikistan has 2 and Kyrgyzstan has 7 only.

DISCUSSION

This analysis of the reviewers from the core CA countries reveals that Kazakhstan seems to top the list while two specific countries need a major boost. The number of active reviewers as well institutes in Kazakhstan reveal its potential as a rising star in the region. Uzbekistan also seems to be doing well. These two countries represent more than 90% of researchers registered on Publons as well as those involved in active peer-reviewing.

Kazakhstan has natural resources and is a major contributor to the economy of CA.14 Similarly, Uzbekistan has been gradually improving its economy and contribution to education in the last decade. These can be helping them made rapid strides in research and publications.4 Similarly, the increase in publications can also influence knowledge of publication ethics and promote peer reviewing practices.

Looking at the performance of their top institutes, Kazakhstan has a good ratio of peer-reviews to publications. This ratio is less for all others including Uzbekistan. Also, the eight reviewers from the region who have received recognition from Publons as top reviewers highlight the point that the potential of this region is second to none. This recognition can also help stimulate their peer to achieve such distinctions.

Peer reviewing not only trains the mind to be analytical but also teaches many aspects of publication ethics. These include understanding plagiarism,15 authorship criteria,16 declaring potential conflicts of interest17 and also the skill of scientific writing. Thus, it has been proposed that peer-reviewing should be a part of ongoing medical education.18

There are different metrics to measure researcher and author impact.13 However, for peer reviewers, they are no other metrics beyond what is present in Publons: the number of journals reviewed for, and the number of verified reviews (overall and in the last 12 months). Also, Publons has an option in which Editor can recommend someone as an excellent reviewer and also, it has its mechanism to award top reviewer recognition.

The most pertinent question is how to mine the potential of Central Asian reviewers. The Publons Academy is an initiation that enables experienced peer reviewers to mentor novice and upcoming reviewers. There is a need to involve both experts on this initiative to devote their time for mentoring and also spread awareness amongst the CA reviewers to get registered for such mentoring. Journal editors can help organize online or offline courses on peer-reviewing for these nations. Online education has added the advantage of bringing together experts from varied regions of the world. Nevertheless, it cannot be a one-way process. The CA countries, especially the underperforming ones, will also need to pull up their bootstraps. There has been freedom of knowledge, an open platform for learning and limitless striving to yearn for perfection. The other countries can take a leaf out of the book from Kazakhstan and stress for more open networking and learning resources.

The leading university of Kazakhstan, the Nazarbayev University, is contributing to half the top reviewers in the region, as well as the highest proportion of verified reviews. Thus, it has an unparalleled capacity for the region and can take an initiative to bridge across to other universities in the region and disseminate their expertise.

There are limitations to this analysis. Firstly, some active reviewers may not be registered on Publons. All registered reviewers also might not be active on Publons. Secondly, there may be predatory journals also registered on Publons and all the peer reviews verified may not have any minimum quality assurance.19

There seems to be no practical way to include reviewers not on Publons, in our analysis. We presume they would be in equal proportion in each country. However, a greater proportion may be from Tajikistan and Turkmenistan possible due to reduced awareness about Publons. The CA countries have different access to the internet and social media.20 Also, Publons is evolving and trying out different strategies to exclude fake or poor reviews.12

Thus, there is heterogeneity in the distribution of peer reviewer in the CA countries. There seems to be a good number of upcoming reviewers in a couple of countries while the others may need more awareness. This region may benefit much from international networking, symposia and e-learning platforms dedicated to promoting peer-reviewing.