Objavljeno u: P. Šipka (Ed.), Journal Publishing in Developing, Transition and Emerging Countries: Proceedings of the 5th Belgrade International Open Access Conference 2012, Belgrade, Serbia, May 18-19th, 2012, pp. 37-48, Belgrade: Centre for Evaluation in Education and Science
"Face Validity" differences between locally published and highest level international journals based on in-text citations approach
University of Novi Sad, Faculty of Philosophy, Dpt. of Psychology, Novi Sad,
NGO “Persona”, Banjaluka (Bosnia and Herzegovina)
Sažetak: Locally published journals are often in a disadvantage when being compared to the highest level international journals using popular approaches such as the impact factor. This research explores an alternative approach, which focuses on the internal article’s “face validity” characteristics, based on the in-text citations measures. The sample comprised 157 empirical articles (psychology = 65, kinesiology = 32, psychiatry = 26, pedagogy = 23, and others = 11) from the journals published in Serbia in the period from 2004 to early 2010, which were compared to the matched 157 articles from the highest level international (English language) journals. ULS factor analysis conducted on the various in-text citations variables revealed two factors: Integration (containing variables such as: number of citations and references, number of citations that occur in multiple sections etc.), and Superficiality (containing: missing, imprecise and secondary citations). Large differences were obtained between the groups on both factors: Integration was significantly higher in the international group (d = 1.49), and Superficiality in the Serbian group (d = 1.10). Ward’s cluster analysis conducted on the two factors’ scores revealed that the articles can be classified in two clusters, with the first containing 117 international, and 9 Serbian articles (psychology = 4, psychiatry = 2, kinesiology = 2, and pedagogy = 1), and the second containing 148 Serbian, and 40 international articles (kinesiology = 16, psychiatry = 12, psychology = 5, pedagogy = 4, and others = 3), with the cluster distribution difference between the local Serbian and international articles being significant: χ² (1, n = 314) = 154.61, p <.001, Ï• = -.70. Strong differences between the clusters on both factors were obtained, showing higher scores on Integration in cluster one (d = 2.14), and higher scores on Superficiality in cluster two (d = 0.93), thus the clusters were named as a Higher tier and a Lower tier (of articles). It should also be noted that there is a certain trend of improvement in Serbian articles on the Integration dimension over the observed time span (ρ =.29, p <.001). In general, the results confirm that the in-text citations based approach, and subsequently derived latent dimensions can be used to differentiate locally published and the highest level international journal articles across various scientific disciplines, and that on the average, most of the Serbian articles fall behind most of the international articles, both on the grounds of internal citation integration, and citation superficiality and mistakes. These findings directly suggest the areas of improvement for journals’ editorial boards, reviewers, and authors.
Ključne reči: local journals; journal quality; citation analysis; face validity; social sciences; Serbia; factor analysis; cluster analysis; impact factor
