This study examines whether large language models (LLMs) can provide consistent content-based classification of core Library and Information Science (LIS) journals.
Using 4,405 abstracts from ten core LIS journals (2019–2024), four LLMs – Claude 3 Haiku, GPT-4o Mini, Gemini Flash 2.0 and DeepSeek Coder Instruct – were prompted to classify each article into one of three categories: Core (central to the LIS field), CorePeriph (partially aligned with core themes) or Peripheral (loosely related or interdisciplinary). Inter-model agreement and journal-level classifications were then analyzed using statistical measures.
Library-centered journals were consistently classified as Core, while interdisciplinary journals like the Journal of the Association for Information Science and Technology were often labeled CorePeriph. However, inter-model agreement varied substantially (0.49–0.88). LLMs reproduced rather than resolved existing uncertainties about disciplinary boundaries, with their disagreements mirroring broader scholarly debates about field identity. This suggests computational approaches face the same epistemological limitations as human-based classification systems.
This study is limited to ten LIS journals and abstract-only analysis. Model disagreements may reflect the evolving nature of interdisciplinary boundaries rather than technical limitations.
This study demonstrates that LLMs reproduce rather than resolve disciplinary boundary uncertainties, revealing fundamental limitations in computational approaches to defining academic fields.
