

The Concepts of Indicator and Criterion in Scientific Research (Towards the 40th Anniversary of an Article by V.M. Motylev)
https://doi.org/10.25281/0869-608X-2025-74-2-179-190
Abstract
V.M. Motylevʼs 1985 article entitled “The Content and Correlation of the Concepts of ‘Indicator’ and ‘Criterion’” is considered as well as a number of his other published work close in time and content. It is shown that V.M. Motylevʼs interpretation and definition of these concepts in relation to the disciplines of socio-communication cycle and the so-called metric research, objectively remains today the best and more preferable ones to be used than the definitions of indicator and criterion in various dictionaries. V.M. Motylev gave the following definitions: “...an indicator is some measurable property of an object used to quantitatively characterize another property that is not directly measurable”; “...in quantitative research a criterion is an evaluative indicator, the value of which is perceived as a numerical expression of the qualitative ratio ‘better — worse’ when making managerial decisions. The larger (or smaller) the value of the criterion is, the better is considered to be the variant of the state or property of the object corresponding to it”. Thus he established that “the formation of a quantitative criterion comes down to selecting one of many indicators describing the properties of the object to be used in the evaluation mode”. The authorsʼ check of definitions and use of these terms in specialized and non-specialized dictionaries, as well as in domestic and foreign publications has shown that in most cases there is a complete disregard of V.M. Motylevʼs formulations, which facilitates to occurrence of confusion, and complete impossibility of comparing the results of different studies, for which, inter alia, the unification of scientific terminology serves. So far, studies positioning themselves as “metric” regularly limp because of confusion in the concepts of “measurement” and “quantitative research”, directly measured and indirectly assessed property of the object (or “indicator” and property under assessment), “indicator” and “criterion”. All this makes it difficult to conduct such research, although V.M. Motylev wrote 40 years ago that “quantitative analysis consists not in ... calculations, but in identifying relationships between phenomena, determining the form and closeness of these relationships, revealing trends and patterns”.
About the Authors
Viacheslav M. TyutyunnikRussian Federation
Moscow State Institute of Culture,
7 Bibliotecnaya Str., Khimki, 141406, Russia
Tambov State Technical University,
116 Sovetskaya Str., Tambov, 392000, Russia
International Nobel Information Center,
30-6 Pervomayskaya Sq., Tambov, 392002, Russia
Belarusian National Technical University,
65 Independence Ave., Minsk, 220013, Belarus
ORCID 0000-0003-2099-5730; SPIN 1442-1686
Vladimir S. Lazarev
Belarus
65 Independence Ave., Minsk, 220013, Belarus
ORCID 0000-0003-0387-4515; SPIN 9750-7985
References
1. Motylev V.M. Content and Correlation of the Concepts “Indicator” and “Criterion”, Sovetskoe bibliotekovedenie [Soviet Library Science], 1985, no. 1, pp. 14—23. Available at: https://rusneb.ru/catalog/000199_000009_011085034 (accessed 18.02.2025) (in Russ.).
2. Chubaryan O.S. (ed.) Slovar’ bibliotechnykh terminov [Dictionary of Library Terms]. Moscow, Kniga Publ., 1976, 222 p.
3. Terminologicheskii slovar’ po informatike [Dictionary of Computer Terms]. Moscow, 1975, 752 p.
4. Suslova I.M., Ulanova L.N., Kartashov N.S. (eds.) Bibliotechnoe delo: terminologicheskii slovar’ [Librarianship: A Terminological Dictionary]. Moscow, Kniga Publ., 1986, 223 p.
5. Fokeev V.A. Bibliograficheskaya nauka i praktika: terminologicheskii slovar’ [Bibliographic Science and Practice: A Terminological Dictionary]. St. Petersburg, Professiya Publ., 2008, 269 p.
6. Ratnikova E.I., Starodubova N.Z., Stolyarov Yu.N. (eds.) Bibliotechnyi fond: slovar’-spravochnik [Library Collection: A Dictionary-Reference Book]. Moscow, INFRA-M Publ., 2016, 158 p.
7. Motylev V.M. The Issues of Quantitative Research in Library Science, Problemy tekhnicheskogo perevooruzheniya bibliotek [The Problems of Technical Re-Equipment of Libraries]. Leningrad, 1983, pp. 55—64 (in Russ.).
8. Motylev V.M. Osnovy kolichestvennykh issledovanii v bibliotechnoi teorii i praktike [Principles of Quantitative Research in Library Theory and Practice]. Leningrad, Nauka Publ., 1988, 196 p.
9. Lazarev V.S. Bibliometrics, Scientometrics and Informetrics. Part 1. Emergence and Background, Upravlenie naukoi: teoriya i praktika [Science Management: Theory and Practice], 2020, vol. 2, no. 4, pp. 133—163. DOI: 10.19181/smtp.2020.2.4.6 (in Russ.).
10. Bornmann L., Daniel H.-D. What Do Citation Counts Measure? A Review of Studies on Citing Behavior, Journal of Documentation, 2008, vol. 64, no. 1, pp. 45—80. DOI: 10.1108/00220410810844150.
11. Tahamtan I., Bornmann L. What Do Citation Counts Measure? An Updated Review of Studies on Citations in Scientific Documents Published Between 2006 and 2018, Scientometrics, 2019, vol. 121, no. 3, pp. 1635—1684. DOI: 10.1007/s11192-019-3243-4.
12. Waltman L., Traag V.A. Use of the Journal Impact Factor for Assessing Individual Articles: Statistically Flawed or Not? F1000Research, 2021, 9:366. DOI: 10.12688/f1000research.23418.2.
13. Molas-Gallart J., Ràfols I. Why Bibliometric Indicators Break Down: Unstable Parameters, Incorrect Models and Irrelevant Properties, BiD: Textos Universitaris de Biblioteconomia i Documentació, 2018, no. 40 (June). DOI: 10.1344/BiD2018.40.23.
14. Diodato V. Dictionary of Bibliometrics. New York, The Haworth Press Publ., 1994, xiii, 185 p.
15. Lazarev V.S. On the Value of a Scientific Document. Part 2, Upravlenie naukoi: teoriya i praktika [Science Management: Theory and Practice], 2024, vol. 6, no. 1, pp. 165—197. DOI: 10.19181/smtp.2024.6.1.9 (in Russ.).
16. Ozhegov S.I., Shvedova N.Yu. (eds.) Indicator, Tolkovyi slovar’ russkogo yazyka [The Explanatory Dictionary of the Russian Language]. Moscow, 1997, pp. 549.
17. Evgenieva A.P. (ed.) Dictionary of the Russian Language: in 4 volumes, Russkaya literatura i fol’klor: fundamental’naya ehlektronnaya biblioteka [Russian Literature and Folklore: Fundamental Electronic Library]. Moscow, Russian Language: Polygraphresursy, 1999, vol. 3. Available at: https://feb-web.ru/feb/mas/mas-abc/default.asp (accessed 20.02.2025) (in Russ.).
18. Indicator, Cambridge Dictionary. Available at: https://dictionary.cambridge.org/dictionary/english/indicator (accessed 20.02.2025).
19. Indicator, Merriam-Webster Dictionary. Available at: https://www.merriam-webster.com/dictionary/indicator (accessed 20.02.2025).
20. Stolyarov Yu.N. Kriterii otsenki bibliotechnogo obsluzhivaniya: uchebnoe posobie [Criteria for Evaluating Library Services: textbook]. Moscow, 1982, 79 p.
21. Scales P.A. Citation Analyses as Indicators of the Use of Serials: A Comparison of Ranked Title Lists Produced by Citation Counting and from the Use Data, Journal of Documentation, 1976, vol. 32, no. 1, pp. 17—25. DOI: 10.1108/eb026612.
22. Mikhailov O.V. Citation Index of a Scientist: Is It the Most Important Criterion of the Quality of the Scientific Activity? Naukovedenie, 2001, no. 1, pp. 201—207 (in Russ.).
23. Lazarev V.S. Readers’ Appeals to Scientific Periodicals as an Indicator of Thematic Orientation of Publications, Metodologicheskie problemy meditsinskoi informatiki i naukovedeniya: sbornik nauchnykh trudov [Methodological Problems of Medical Informatics and Science Studies: Collected Articles]. Moscow, 1989, pp. 173—186 (in Russ.).
24. Lazarev V.S. Properties of Scientific Periodicals Under Bibliometric Assessment, International Journal of Information Sciences for Decision Making, 1997, no. 1 (December), pp. 1—17. Available at: https://isdm.univ-tln.fr/PDF/isdm1/isdm1a6_lazarev.pdf (accessed 20.02.2025).
25. Lazarev V.S. Scientific Documents and Their Structured Collections: Citation, Use, Value, Mezhdunarodnyi forum po informatsii [International Forum on Information], 2017, vol. 42, no. 1, pp. 3—16 (in Russ.).
26. Gilyarevsky R.S. Informatsionnaya sfera: kratkii ehntsiklopedicheskii slovar’ [Information Sphere: A Concise Encyclopedic Dictionary]. St. Petersburg, Professiya Publ., 2016, 302 p.
27. Zakharchuk T.V., Zusman O.M. (eds.) Search and Dissemination of Information. Terms and Definitions. GOST 7.73—96, Standarty po bibliotechno-informatsionnoi deyatel’nosti [Standards for Library and Information Activities]. St. Petersburg, Professiya Publ., 2003, pp. 307—317 (in Russ.).
28. Vinokur G.O., Larin B.A., Ozhegov S.I., Tomashevsky B.V., Ushakov D.N. (eds.) Criterion, Tolkovyi slovar’ russkogo yazyka: v 4 t. [The Explanatory Dictionary of the Russian Language: in 4 volumes]. Moscow, Sovetskaya Ehntsiklopediya Publ.: OGIZ, 1935, vol. 1, col. 1518 (in Russ.).
29. Measure, Tolkovyi slovar’ russkogo yazyka: v 4 t. [The Explanatory Dictionary of the Russian Language: in4 volumes]. Moscow, Gosudarstvennoe Izdatel’stvo Inostrannykh i Natsional’nykh Slovarei, 1938, vol. 2, col. 186 (in Russ.).
30. Criterion, Dictionary.com. Available at: https://www.dictionary.com/browse/criterion (accessed 20.02.2025).
31. Criterion, Cambridge Dictionary. Available at: https://dictionary.cambridge.org/dictionary/english/criterion (accessed 20.02.2025).
When quantifying properties that are not directly measurable, it is often unclear what exactly is being evaluated and how adequate the evaluation is. Therefore, a use of clear-cut terminology is a prerequisite for the successful conduct of such an evaluation. However, currently it is useless to look for well-defined normative definitions and interpretations of the concepts of “indicator” and “criterion” in relation to the so-called metric research and quantitative research in the disciplines of the social-and-communication cycle. Meanwhile, there is an interpretation of them that is promising for use and brings clarity to the conceptual framework. It was proposed 40 years ago on the pages of this very journal by V.M. Motylev, whose relevant article is reviewed in this paper.
Review
For citations:
Tyutyunnik V.M., Lazarev V.S. The Concepts of Indicator and Criterion in Scientific Research (Towards the 40th Anniversary of an Article by V.M. Motylev). Bibliotekovedenie [Russian Journal of Library Science]. 2025;74(2):179-190. (In Russ.) https://doi.org/10.25281/0869-608X-2025-74-2-179-190