2010, vol. 44, iss. 2, pp. 283-308
Hot rhetorics and cool reality: A contribution to the testing of the general sociometric-scientometric hypothesis
Faculty of Philosophy in Novi Sad, Department of Psychology,
Centre for Evaluation in Education and Science, Belgrade
Abstract: The background of the extremely hostile rhetoric of a small group of local ethnologist in their alleged criticism of using scientometrics in research evaluation in Serbia was studied using publicly available data. Assuming absurdity of the 'criticism' is self-evident, the study was focused around their latent motivation, rather than manifest content of the accusations. Taking as starting point the position of person who was recognized as the inspirer of 'criticism', the citation network of sociocognitively related authors was identified and mapped by using multidimensional scaling. Network partitioning based on successive introduction of various criteria resulted in four groups with centrally positioned Core, containing Hard Core in the middle. Citation rate and citation habits of the Hard Core, were analyzed in comparison with other network groups and suitable controls. To complete the Core portrait, methodological character of the papers appearing in a journal they mainly publish in, as well as the content of the university methodological syllabi they are responsible for, was analyzed, also comparatively. The results revealed the explosive growth of the citation rate of the group, particularly evident for the Hard Core. The growth is based on excessive (instrumental, manipulative) mutual citation exchange, taking mostly place in the two journals published by the group's affiliated institution. The two prominent group members making Hard Core and profiting most from overcitation, who are at the same time the 'critics' of scientometrics, are found to be indulged also in other forms of misconduct, characteristic of social climbers inflating their research performance. Based on the results, a few hypothetical explanations for their fabricated criticism were offered. Regardless of the viability of these explanations the results suggest that citation behavior can be motivated by social group reasons, supporting general scientometric-sociometric hypothesis, and jeopardizing citation rate as an indicator. On the other hand, the methods used show that such misbehaviors are detectable, while their ill consequences are manageable.
Keywords: scientometrics, sociometrics, research performance, journal evaluation, scientific criticism, publication standards, ethics of science