Is Implicit Knowledge Enough for LLMs? A RAG Approach for Tree-based Structures

Large Language Models (LLMs) are effective at generating responses using contextual information, which is especially useful when working with structured data like code. Retrieval-Augmented Generation (RAG) enhances this capability by retrieving relevant documents to supplement the model’s context. However, the best way to represent retrieved knowledge—particularly for hierarchical structures such as trees—remains underexplored. In this work, we introduce a novel method to linearize tree-structured knowledge, making it suitable for storage in a knowledge base and direct use with RAG. We compare this approach to applying RAG on raw, unstructured code, evaluating both the accuracy and quality of the generated responses. Our results show that while response quality is comparable between the two methods, our linearized approach reduces the number of vector documents by almost 4x. Without any loss of generality, we applied and benchmarked our method to a specific automotive use case, that is to generate information from a repository of digital engineering assets, in our case, Simulink based simulation code. This suggests that leveraging implicit, structured representations can be a highly efficient strategy for handling complex hierarchical data which are common for Automotive based information-retrieval applications.