Preview

Scholarly Research and Information

Advanced search

On relations in science: the case of the scientific journal editorial board

https://doi.org/10.24108/2658-3143-2021-4-1-2-19-28

Abstract

This article analyses the connections between members of the editorial board of a scientific journal which are formed based on their reviews of scientific articles. It is shown that the connections can be represented as a graph. The research uses the data for six years of article reviewing in the scientific and technical Journal of "Almaz — Antey" Air and Space Defence Corporation.

The methods of analysis are combinatorics and graph theory, as well as the relevant graph characteristics: adjacency matrix, incidence matrix, reachability matrix, graph fullness and connectivity, nearest neighbours graph and graph spanning tree.

It is shown that cooperation of the reviewers of the Journal helps plot a connected graph with links between any two vertices, i.e. between any reviewers.

The graph is analysed and the methods of its application to calculate the Journal’s scientometric indicators are demonstrated. As the research reveals, a journal that publishes articles in numerous disciplines forms connections between all reviewers and this parameter can be indicative of interchangeability within the scientific fields or, conversely, of a joint work in this scientific field when reviewing manuscripts. Based on the research results, it is possible to search for new reviewers in the areas where competencies are underdeveloped. And by the areas where competencies are strong, we can determine the core competence of the reviewers of a scientific journal, reflecting the main focus of the evaluated scientific research.

The work resulted in the formulated scientometric indicators of the journal, which can be used to search for and involve new reviewers or to represent data on a strong team of reviewers on a specific topic, as well as on a new scientific field just emerging for research.

For citation:


Bolshakov D.Yu. On relations in science: the case of the scientific journal editorial board. Scholarly Research and Information. 2021;4(1-2):23-32. https://doi.org/10.24108/2658-3143-2021-4-1-2-19-28

Introduction

So far, publication citations are supposed to be the basic criterion for estimating connections in science. However, such connections can be established between scientists in the course of expert review of manuscripts prior to publication in a scientific journal. Reviewing research papers helps establish the connections between scientists both from allied sciences and from absolutely different fields. This is an indirect proof of certain connections between different branches of science.

Moreover, scientific connections being established between reviewers allow to understand the research fields of a particular scientific journal and analyse scientific disciplines where reviewers can replace one another in validation of manuscripts.

We should note that some research papers are not published in the journal, because certain percentage of manuscripts is rejected or returned for revision. However, within a journal, connections between scientists are also established through co-reviewing of an unpublished manuscript. This allows to evaluate the science from the other side — it is not a research itself that is analysed, but review of the research papers that are not published. Further, this paper is not intended to draw a line between a published or unpublished manuscript. The goal is to analyse relevant connections formed between scientists.

Researchers have already applied the methods described herein and related to the graph theory in order to analyse a science and its allied fields [1–8]. For example, an analysis of reviewer’s work in editorial boards of several scientific journals and reviewer’s connections with research activities at a higher education institution are described in [1]. The graph theory along with the graph connectivity used in this research is employed to identify relations in conflict resolution studies [2]. The graph theory is proposed for visualization in the cognitive science [3]; research papers [4][5] describe the potential of the graph theory in information visualization. An overview of papers devoted to the scientometric research of editorial boards of scientific journals is given in [6]. A study in the field near to this research is given in [7], devoted to a graph theory-based analysis of the network of scientific journals and scientists, as well as the relevant united groups. Paper [8] analyses the influence of academic journals and their editorial boards on research fields and describes the formation of a social network consisting of interconnected editorial boards of scientific journals.

A graph theory-based analysis of connections between reviewers

Table 1 shows the frequency of assignment of a particular number of reviewers to articles submitted to the scientific and technical Journal of “Almaz — Antey” Air and Space Defence Corporation for the period of 2015–2021. The reference year is 2015, because since the second half of that year the journal’s policy was changed to send an article to more than one reviewer.

Table 1Frequency of assigning the number of reviewers to articles by year

Number of reviewers (reviews) for articles

Year

2015

2016

2017

2018

2019

2020

2021*

0

6

2

1

11

2

8

5

1

53

33

84

36

41

24

10

2

5

22

22

25

38

35

16

3

1

8

11

16

27

17

12

4

1

5

4

10

9

7

6

5

1

1

3

3

1

1

1

6

7

1

1

1

8

1

Average number of reviewers per article

1,1

1,8

1,5

1,9

2,1

2,0

2,2

*2021 data represent the first half of the year only

As shown in Table 1, the maximum number of reviews distributed from one to two and more reviewers was shifted within the period of 2019 to 2020. This tendency continues in 2021.

It should be noted that articles that have failed to pass the check by the Antiplagiat plagiarism detection system are returned to authors unreviewed, with no reviewer assigned (number of reviews is 0).

Moreover, articles that do not meet the journal’s requirements to documentation execution or those of a very low scientific value are rejected and returned to authors. In this case, only one reviewer is assigned (1 review per article). As stated in [9], conversion of articles, i.e. the ratio between the number of articles accepted for publications and the total number of submitted articles is a random value with normal distribution, which can be substituted with an average value equal to 0.4 (or 40 %) for the scientific and technical Journal of “Almaz — Antey” Air and Space Defence Corporation. In other words, one reviewer can be assigned to review 60 % of submitted articles, the quality of which fails to comply with the publication requirements.

A graph is plotted based on the relations forming between two and more reviewers working on an article [10].

This graph is plotted as follows. Assume that a journal contains only two articles, and three reviewers are assigned to review one of them (Reviewer 1, Reviewer 2, and Reviewer 3), while two reviewers are assigned to review the other one (Reviewer 3 and Reviewer 4). After all the reviewers issue their conclusions of the first paper, links between reviewers will be established in relation to this paper as shown in the left part of Fig. 1. As for the second article, links are established in the right part of the graph between Reviewer 3 and Reviewer 4 after reviewers draw their conclusions. However, Reviewer 3 has connections with Reviewer 1 and Reviewer 2, while Reviewer 4 has no such links.

Fig. 1. An example of constructing a graph of reviewers relations

We should note that the reviewing routine process at the scientific and technical Journal of “Almaz — Antey” Air and Space Defence Corporation includes a smooth-running review exchange between experts involved in manuscript evaluation, but only after they present their expert opinions. Besides, journal articles cover different areas of science and engineering such as automation, aerodynamics, hydraulics, gas and fluid dynamics, mechanics, radio location, radio navigation, radio electronics, radio engineering, power electronics, thermal physics, chemical technologies, electric engineering, and electrodynamics. The journal’s editorial board includes many researchers studying not only their own field, but a few allied sciences (for instance, automation and hydraulics, aerodynamics and thermal physics, etc.).

If there is a single reviewer, he or she is not considered in graph plotting; if there are two and more reviewers, they are considered in plotting the connection with the number of possible combinations taken into account as well. For example, with three reviewers (1, 2, 3), all possible link combinations are considered (1-2, 2-3, 3-1). Directions are not considered, i.e. link 1-2 is identical to link 2-1. The number of combinations Сn² is calculated by the known formula [11][12]

(1)

The total number of combinations calculated by formula (1) is given in Table 2.

Table 2The number of relations of reviewers from their number

Parameter

Meaning

Number of reviewers per article

2

3

4

5

6

7

8

Number of combinations (relations)

1

3

6

10

15

21

28

Research results

As a result, with 117 reviewers for the scientific and technical Journal of “Almaz — Antey” Air and Space Defence Corporation taken for research, 503 unique links have been identified (910 links result from 318 records of reviews made by several reviewers). Fig. 2 illustrates the graph linking 117 reviewers via 503 unique link combinations. The graph is plotted using the Graph[] function in the Wolfram Mathematica, a computer algebra system. Graph visualization is made with the help of the GraphPlot[] function, links between reviewers are identified using the Ksubsets[] function that generates combinations from a data array. The graph shown in Fig. 2 is modified in the graphic editor Adobe Illustrator in order to adjust its sizes and improve visualization. To see the detailed study, click the following link to an online platform https://www.wolframcloud.com/env/3f0eb203-a684-4122-8554-b7a24dbdc986 or use a QR code below.

Fig. 2Graph of relations within the editorial board of the scientific Journal of “Almaz — Antey” Air and Space Defence Corporation

Red colour indicates those reviewers who no longer cooperate with the journal’s editorial board for some reasons. For example, Reviewer 90 died, Reviewer 81 was unable to cooperate due to lack of time, being involved in other projects, and asked not to send manuscripts for reviewing. The graph is plotted with no consideration given to those reviewers who did not participate in reviewing of submitted articles.

The graph shown in Fig. 2 allows:

  1. To identify those reviewers who review a wide range of articles based on a large number of connections with other reviewer (Reviewer 16 has 38 connections, Reviewer 84 – 24 connections).
  2. To estimate the most frequently reviewed research areas, i.e. concentration zones which have a large amount of links and a high concentration of reviewers, or competences (“Electronics. Radio Engineering” – left part of the graph, “Space Research and Rocket Science” – right part of the graph).
  3. To determine research areas that are reviewed in the journal on rare occasions and, therefore, to determine reviewers whose research interests lie in quite a narrow area overlapping with the journal’s subject areas (Reviewer 40 “Radiology”). To single out the growth zones, i.e. graph vertices which have a small number of connections, probably meaning a small number of reviewers, or that such areas are rarely covered in the journal.
  4. To form a reviewers’ interchangeability matrix based on the connection graph.
  5. To emphasize that most reviewers, who have failed to build good work relationships with the editorial board, terminate their cooperation within a short period of time and do not form an appropriate number of connections in a given direction (no more than 3).
  6. To calculate scientometric indicators based on the plotted graph of connections between reviewers (for example, full interchangeability of reviewers indicates that a journal focuses on special research areas only; graph disconnectedness indicates disconnectedness of research papers published in a journal).

We should note that the full graph illustrates an ideal case — all reviewers are interconnected. For the case with 117 reviewers, the graph has 6,786 edges (formula n(n–1)/2) or connections between reviewers [11][12]. This means that all reviewers are absolutely interchangeable. In fact, it is possible only if a journal focuses on very special disciplines, for example, on publications related to a single discipline as per the HAC list. If a graph plotted on the basis of reviews received from reviewers is not connected and has two or more no longer connected sets of vertices and edges, such a journal can be classified as an interdisciplinary journal (for example, philology and biophysics). However, even for a multidisciplinary journal, with disjoint graphs illustrating connections between members of the editorial boards, relations between these connections and the connected graph can be established in order to determine the reviewers’ interchangeability depending on the direction.

Another graph (Fig. 3) is plotted based on the analysis of the adjacency matrix. The graph analyses the length of connections between reviewers. Since this graph is a connected one, there is a path between any two vertices.

Fig. 3. The number of path lengths between vertices in the graph (between reviewers) along the ordinate axis and the path length along the abscissa axis

As shown in Fig. 3, the total amount of the longest paths between the members of the editorial board is not greater than 6. This is a proof of the six degrees of separation theory, which states that everyone in the world is connected by just six acquaintances [13]. The zero path length corresponds to the number of graph vertices, i.e. to the number of reviewers, and is equal to 117 as expected. We should emphasize again that the resulting graph for the journal under consideration is a connected graph with all the vertices connected with one another. The path length indicates the closeness of research activities within an organization and the connectivity of research findings.

The number of connections between reviewers and their colleagues can be calculated by plotting a spanning tree based on the nearest neighbors’ graph, starting from the vertex with a certain reviewer number, as shown in Fig. 4 for Reviewer 32 (“Thermal physics”).

(а)
(b)

Fig. 4. The nearest neighbor graph for reviewer No. 32 (a) and the construction of a spanning tree on its basis (b)

Plotting a spanning tree based on the nearest neighbours graph (a connected acyclic graph with the number of vertices similar to that of the initial graph with the number of edges equal to the number of vertices reduced by one, with the central vertex degree equal to the number of edges, and with the degree of other vertices equal to one; the incidence matrix of such a graph contains unities in the row with the central vertex, unities in the column of edges incident to a given vertex, and zeros in all other elements of the incidence matrix) allows to estimate the number of reviewer’s connections with other reviewers [10]. As shown in Fig. 4b, it is easy to calculate the number of connections of Reviewer 32, which is equal to 13. Fig. 5 shows the number of connections for all 117 reviewers calculated as demonstrated above.

Fig. 5. The number of connections of reviewers along the abscissa axis and the frequency of occurrence of this event along the ordinate axis

Discussion of results

As shown in Fig. 5, the maximum number of connections between reviewers is 1–5 with further decrease. The more connections a reviewer has with other reviewers within the editorial board, the higher the reviewer’s competence in reviewing manuscripts related to the specific research area or to allied sciences. It should be noted that a reviewer making poor-quality reviews is removed from the editorial board of the scientific and technical Special Journal of “Almaz — Antey” Air and Space Defence Corporation.

Those reviewers who are located beyond the centre of the graph and have a small number of connections with other reviewers, can belong to three types:

  • reviewers focused on a limited field of science (for instance, Reviewer 40 — “Radiology”);
  • reviewers recently included in the reviewer staff of the journal (Reviewer 91 — “Optoelectronic Systems and Drives”);
  • reviewers who failed to build good working relationships with the editorial board (Reviewer 110).

For example, assume that the journal under consideration publishes articles related to the only discipline — “Electronics. Radio Engineering”. In this case, the graph contains only those reviewers who review articles related to the specified subject (Reviewers 6, 8, 14, 17, 20, 21, 54, 71, 79, 80, 81, 90, 100, 103, 107). Except the connections with other reviewers, calculation of the number of connections between them (Fig. 6a) in relation to the number of edges of the full graph gives the value equal to 0.61 (64/105). This number is less than one, but the author supposes that such a result is obtained due to a short observation time: to form interconnections of all reviewers related to subject “Electronics. Radio Engineering”, in case of 15 reviewers, at least 105 articles shall be reviewed.

Reviewers specialising in subject “Gas Dynamics” (Reviewers 25, 74, 60) form three connections with one another, as clearly shown in Fig. 6b: the ratio between the number of the connections and the number of edges of the full graph for three vertices gives the exact value equal to 1.

а)(b)

Fig. 6. Reviewers in the direction of “Electronics. Radio Engineering” (a) and “Gasdynamics” (b) and their connections

Data given in Table 2 can be used as follows: if a journal publishes articles related to several disciplines and its reviewers are not connected in terms of the discipline to be covered, it is reasonable to divide a journal in two or more, provided that articles related to each separated discipline are submitted on a regular basis.

For a limited-field journal, there are connections between all reviewers within the limit, because reviewers have competences to review any material to be published. The connection indicator can be viewed as an indirect estimate of a reviewer and the journal’s editorial board: materials related to a certain subject area are to be reviewed by more than one reviewer. This approach allows to shape different opinions on the materials to be published, because each reviewer has their own opinion regarding this or that research area.

Anyway, materials to be reviewed shall be sent to several reviewers. This works both for multidisciplinary and for limited-field journals. The experience gained by the staff of the scientific and technical Journal of “Almaz — Antey” Air and Space Defence Corporation proves that article processing involving more than two reviewers helps improve a manuscript.

Therefore, the ratio between the number of edges of the reviewers connections’ graph and the number of edges of the full graph with the same number of vertices, the value of which in the limit is equal to one, is the scientometric indicator. For a multidisciplinary journal, the scientometric indicator is the existence of two or more disconnected graphs of connections between reviewers, but within each graph its number of connection in the limit tends to the indicators of a limited-field journal. The scientometric indicator equal to one as well as other indicators (impact factor, Gini index, etc.) can be used as a reference mark for the editorial board of a limited-field journal. For the editorial board, this parameter indicates that all reviewers shall take part in the expert review process, while the limit indicator requires that the number of reviewers per particle shall be at least two, and reviewers shall be rotated from one article to another.

Conclusions

  1. The number of reviewer’s connections indicates that a reviewer is a multi-discipline expert able to review articles related to allied sciences.
  2. The ratio between the number of reviewers’ connections and the number of connections of the full graph for the same number of reviewers indicates journal’ multidisciplinarity or limited field with all reviewers interchangeable.
  3. A field-specific journal enables close connections between reviewers, and the path length in the connection graph is near or equal to one.
  4. A multidisciplinary journal may have no connections between reviewers, and the connection graph of such a journal is not connected.
  5. The length of the longest path in the journal reviewers’ connected graph is not greater than six, thus proving the six degrees of separation hypothesis.
  6. The graph illustrating connections between scientific journal’s reviewers can be used for constructing an interchangeability matrix based on the published articles.
  7. As a rule, a reviewer who fails to get along with the editorial board of a scientific journal, forms a small number of connections with other reviewers before he or she is removed from the editorial board.

Table 3Limiting scientometric indicators for journals of one and several topics

Characteristic

Direction

One

Several

The ratio of the number of edges-connections between reviewers to the number of edges of the complete graph for a given number of vertices

1

-

Related graph of cooperation between reviewers

Yes

No

The number of unrelated graphs of cooperation between reviewers

0

2 and more

The length of the path between the vertices

1

From 1 and more

The number of links of each reviewer

[Number of reviewers] –1

More than 1, but less than the number of reviewers

References

1. Krylova O.S., Krylov D.A. Work in the editorial board of a scientific journal as an indicator of the professional competence of a university teacher. Vestnik Mariiskogo gosudarstvennogo universiteta = Vestnik of the Mari State University. 2018;12(4): 44–50 (In Russ.). https://doi.org/10.30914/2072-6783-2018-12- 4-44-50 (In Russ.).

2. Putkina L.V., Sedov R.L. On the applications of the theory of graphs in conflictology: the experience of the humanitarian university. CITISE. 2019;(1):15 (In Russ.).

3. Morozova O.A. Structural network modelling in cognitive science. Psikhologicheskie issledovaniya = Psychological Studies [Internet]. 2017;10(55):1. Available at: http://psystudy.ru/index.php/num/2017v10n55/1474-morozova55.html (In Russ.).

4. Apanovich Z. V. From Graph Drawing Towards Information Visualization. Preprint Novosibirsk: Siberian Division of the Russian Academy of Sciences A.P. Ershov Institute of Informatics Systems; 2007. 28 p. (In Russ.).

5. Tselykh A.A., Tselykh A.N., Matveev D.A. Methods and techniques for visualizing arrays of scientific and technical indicators using graphs. Sovremennye problemy nauki i obrazovaniya = Modern problems of science and education [Internet]. 2013;(3):69. Available at: https://science-engineering.ru/ru/article/view?id=687 (In Russ.).

6. Mazov N.A., Gureev V.N. The editorial boards of scientific journals as a subject of scientometric research: A literature Review. Sci. Tech. Inf. Proc. 2016; 43(3):144–153. https://doi.org/10.3103/S0147688216030035

7. Goyanes M., de-Marcos L. Academic influence and invisible colleges through editorial board interlocking in communication sciences: a social network analysis of leading journals. Scientometrics. 2020;123(2):791–811. https://doi.org/10.1007/s11192-020-03401-z

8. Teixeira E.K., Oliveira M. Editorial board interlocking in knowledge management and intellectual capital research field. Scientometrics. 2018;117(3):1853–1869. https://doi.org/10.1007/s11192-018-2937-x

9. Bolshakov D.Yu. Analytics in the publishing of a scientific journal. Nauchnyi redactor i Izdatel’ = Science Editor and Publisher. 2020;5(2):102–112 (In Russ.). https://doi.org/10.24069/2542-0267-2020-2-102-112

10. Deo N. Graph Theory with Applications to Engineering and Computer Science (Dover Books on Mathematics). Dover Publications; 2004. 498 p.

11. Bender E.A., Williamson S.G. Foundations of Combinatorics with Applications. Dover Publications; 2006. 480 p.

12. Gnedenko B.V. Theory of Probability. 6th ed. CRC Press; 2020. 520 p.

13. Tadimety P.R. (2015) Six Degrees of Separation. In: OSPF: A Network Routing Protocol. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842- 1410-7_1


About the Author

D. Yu. Bolshakov
«Almaz — Antey» Air and Space Defence Corporation, Joint Stock Company
Russian Federation

Denis Yu. Bolshakov, Candidate of Technical Sciences, Head of the Department of Scientific and Technical Publications and Special Projects of the Office of the Director General

Vereiskaya str., 41, Moscow, 121471



For citation:


Bolshakov D.Yu. On relations in science: the case of the scientific journal editorial board. Scholarly Research and Information. 2021;4(1-2):23-32. https://doi.org/10.24108/2658-3143-2021-4-1-2-19-28

Views: 148


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.


ISSN 2658-3143 (Online)