The Concepts of Indicator and Criterion in Scientific Research (Towards the 40th Anniversary of an Article by V.M. Motylev)

V.M. Motylevʼs 1985 article entitled “The Content and Correlation of the Concepts of ‘Indicator’ and ‘Criterion’” is considered as well as a number of his other published work close in time and content. It is shown that V.M. Motylevʼs interpretation and definition of these concepts in relation to the disciplines of socio-communication cycle and the so-called metric research, objectively remains today the best and more preferable ones to be used than the definitions of indicator and criterion in various dictionaries. V.M. Motylev gave the following definitions: “...an indicator is some measurable property of an object used to quantitatively characterize another property that is not directly measurable”; “...in quantitative research a criterion is an evaluative indicator, the value of which is perceived as a numerical expression of the qualitative ratio ‘better — worse’ when making managerial decisions. The larger (or smaller) the value of the criterion is, the better is considered to be the variant of the state or property of the object corresponding to it”. Thus he established that “the formation of a quantitative criterion comes down to selecting one of many indicators describing the properties of the object to be used in the evaluation mode”. The authorsʼ check of definitions and use of these terms in specialized and non-specialized dictionaries, as well as in domestic and foreign publications has shown that in most cases there is a complete disregard of V.M. Motylevʼs formulations, which facilitates to occurrence of confusion, and complete impossibility of comparing the results of different studies, for which, inter alia, the unification of scientific terminology serves. So far, studies positioning themselves as “metric” regularly limp because of confusion in the concepts of “measurement” and “quantitative research”, directly measured and indirectly assessed property of the object (or “indicator” and property under assessment), “indicator” and “criterion”. All this makes it difficult to conduct such research, although V.M. Motylev wrote 40 years ago that “quantitative analysis consists not in ... calculations, but in identifying relationships between phenomena, determining the form and closeness of these relationships, revealing trends and patterns”.

1. Motylev V.M. Content and Correlation of the Concepts “Indicator” and “Criterion”, Sovetskoe bibliotekovedenie [Soviet Library Science], 1985, no. 1, pp. 14—23. Available at: https://rusneb.ru/catalog/000199_000009_011085034 (accessed 18.02.2025) (in Russ.).

2. Chubaryan O.S. (ed.) Slovar’ bibliotechnykh terminov [Dictionary of Library Terms]. Moscow, Kniga Publ., 1976, 222 p.

3. Terminologicheskii slovar’ po informatike [Dictionary of Computer Terms]. Moscow, 1975, 752 p.

4. Suslova I.M., Ulanova L.N., Kartashov N.S. (eds.) Bibliotechnoe delo: terminologicheskii slovar’ [Librarianship: A Terminological Dictionary]. Moscow, Kniga Publ., 1986, 223 p.

5. Fokeev V.A. Bibliograficheskaya nauka i praktika: terminologicheskii slovar’ [Bibliographic Science and Practice: A Terminological Dictionary]. St. Petersburg, Professiya Publ., 2008, 269 p.

6. Ratnikova E.I., Starodubova N.Z., Stolyarov Yu.N. (eds.) Bibliotechnyi fond: slovar’-spravochnik [Library Collection: A Dictionary-Reference Book]. Moscow, INFRA-M Publ., 2016, 158 p.

7. Motylev V.M. The Issues of Quantitative Research in Library Science, Problemy tekhnicheskogo perevooruzheniya bibliotek [The Problems of Technical Re-Equipment of Libraries]. Leningrad, 1983, pp. 55—64 (in Russ.).

8. Motylev V.M. Osnovy kolichestvennykh issledovanii v bibliotechnoi teorii i praktike [Principles of Quantitative Research in Library Theory and Practice]. Leningrad, Nauka Publ., 1988, 196 p.

9. Lazarev V.S. Bibliometrics, Scientometrics and Informetrics. Part 1. Emergence and Background, Upravlenie naukoi: teoriya i praktika [Science Management: Theory and Practice], 2020, vol. 2, no. 4, pp. 133—163. DOI: 10.19181/smtp.2020.2.4.6 (in Russ.).

10. Bornmann L., Daniel H.-D. What Do Citation Counts Measure? A Review of Studies on Citing Behavior, Journal of Documentation, 2008, vol. 64, no. 1, pp. 45—80. DOI: 10.1108/00220410810844150.

11. Tahamtan I., Bornmann L. What Do Citation Counts Measure? An Updated Review of Studies on Citations in Scientific Documents Published Between 2006 and 2018, Scientometrics, 2019, vol. 121, no. 3, pp. 1635—1684. DOI: 10.1007/s11192-019-3243-4.

12. Waltman L., Traag V.A. Use of the Journal Impact Factor for Assessing Individual Articles: Statistically Flawed or Not? F1000Research, 2021, 9:366. DOI: 10.12688/f1000research.23418.2.

13. Molas-Gallart J., Ràfols I. Why Bibliometric Indicators Break Down: Unstable Parameters, Incorrect Models and Irrelevant Properties, BiD: Textos Universitaris de Biblioteconomia i Documentació, 2018, no. 40 (June). DOI: 10.1344/BiD2018.40.23.

14. Diodato V. Dictionary of Bibliometrics. New York, The Haworth Press Publ., 1994, xiii, 185 p.

15. Lazarev V.S. On the Value of a Scientific Document. Part 2, Upravlenie naukoi: teoriya i praktika [Science Management: Theory and Practice], 2024, vol. 6, no. 1, pp. 165—197. DOI: 10.19181/smtp.2024.6.1.9 (in Russ.).

16. Ozhegov S.I., Shvedova N.Yu. (eds.) Indicator, Tolkovyi slovar’ russkogo yazyka [The Explanatory Dictionary of the Russian Language]. Moscow, 1997, pp. 549.

17. Evgenieva A.P. (ed.) Dictionary of the Russian Language: in 4 volumes, Russkaya literatura i fol’klor: fundamental’naya ehlektronnaya biblioteka [Russian Literature and Folklore: Fundamental Electronic Library]. Moscow, Russian Language: Polygraphresursy, 1999, vol. 3. Available at: https://feb-web.ru/feb/mas/mas-abc/default.asp (accessed 20.02.2025) (in Russ.).

18. Indicator, Cambridge Dictionary. Available at: https://dictionary.cambridge.org/dictionary/english/indicator (accessed 20.02.2025).

19. Indicator, Merriam-Webster Dictionary. Available at: https://www.merriam-webster.com/dictionary/indicator (accessed 20.02.2025).

20. Stolyarov Yu.N. Kriterii otsenki bibliotechnogo obsluzhivaniya: uchebnoe posobie [Criteria for Evaluating Library Services: textbook]. Moscow, 1982, 79 p.

21. Scales P.A. Citation Analyses as Indicators of the Use of Serials: A Comparison of Ranked Title Lists Produced by Citation Counting and from the Use Data, Journal of Documentation, 1976, vol. 32, no. 1, pp. 17—25. DOI: 10.1108/eb026612.

22. Mikhailov O.V. Citation Index of a Scientist: Is It the Most Important Criterion of the Quality of the Scientific Activity? Naukovedenie, 2001, no. 1, pp. 201—207 (in Russ.).

23. Lazarev V.S. Readers’ Appeals to Scientific Periodicals as an Indicator of Thematic Orientation of Publications, Metodologicheskie problemy meditsinskoi informatiki i naukovedeniya: sbornik nauchnykh trudov [Methodological Problems of Medical Informatics and Science Studies: Collected Articles]. Moscow, 1989, pp. 173—186 (in Russ.).

24. Lazarev V.S. Properties of Scientific Periodicals Under Bibliometric Assessment, International Journal of Information Sciences for Decision Making, 1997, no. 1 (December), pp. 1—17. Available at: https://isdm.univ-tln.fr/PDF/isdm1/isdm1a6_lazarev.pdf (accessed 20.02.2025).

25. Lazarev V.S. Scientific Documents and Their Structured Collections: Citation, Use, Value, Mezhdunarodnyi forum po informatsii [International Forum on Information], 2017, vol. 42, no. 1, pp. 3—16 (in Russ.).

26. Gilyarevsky R.S. Informatsionnaya sfera: kratkii ehntsiklopedicheskii slovar’ [Information Sphere: A Concise Encyclopedic Dictionary]. St. Petersburg, Professiya Publ., 2016, 302 p.

27. Zakharchuk T.V., Zusman O.M. (eds.) Search and Dissemination of Information. Terms and Definitions. GOST 7.73—96, Standarty po bibliotechno-informatsionnoi deyatel’nosti [Standards for Library and Information Activities]. St. Petersburg, Professiya Publ., 2003, pp. 307—317 (in Russ.).

28. Vinokur G.O., Larin B.A., Ozhegov S.I., Tomashevsky B.V., Ushakov D.N. (eds.) Criterion, Tolkovyi slovar’ russkogo yazyka: v 4 t. [The Explanatory Dictionary of the Russian Language: in 4 volumes]. Moscow, Sovetskaya Ehntsiklopediya Publ.: OGIZ, 1935, vol. 1, col. 1518 (in Russ.).

29. Measure, Tolkovyi slovar’ russkogo yazyka: v 4 t. [The Explanatory Dictionary of the Russian Language: in4 volumes]. Moscow, Gosudarstvennoe Izdatel’stvo Inostrannykh i Natsional’nykh Slovarei, 1938, vol. 2, col. 186 (in Russ.).

30. Criterion, Dictionary.com. Available at: https://www.dictionary.com/browse/criterion (accessed 20.02.2025).

31. Criterion, Cambridge Dictionary. Available at: https://dictionary.cambridge.org/dictionary/english/criterion (accessed 20.02.2025).