Utilizing structural metrics from knowledge graphs to enhance the robustness quantification of large language models (extended abstract)

Abstract

The goal of this study is to determine whether large language models (LLMs) like CodeLlama, Mistral, and Vicuna can be used to build knowledge graphs (KGs) from textual data. We create class descriptions for well-known KGs such as DBpedia, YAGO, and Google Knowledge Graph, from which we extract RDF triples and enhance these graphs using different preprocessing methods. Six structural quality measures are used in the study to compare the constructed and existing KGs. Our results demonstrate how important LLMs are to improving KG construction and provide insightful information for KG construction researchers. Moreover, an in-depth analysis of popular open-source LLM models enables researchers to identify the most efficient model for various tasks, ensuring optimal performance in specific applications.