Aurel_AI: Automating an Institutional Help Desk Using an LLM Chatbot

Diego Ordóñez-Camacho, Rafael Melgarejo-Heredia, Mohsen Abbasi, Lucía González-Solis

The Aurel_AI research project was born from the need to implement a virtual help desk for a university, providing accurate organizational information to both internal and external clients. The information includes details about academic programs, regulations, processes, and personnel. Aurel_AI is part of a broader research program on the use of AI in academia. Traditional solutions for a help desk, such as telephone call centers, present quality and efficiency issues that are difficult to solve. Call center staff generally lack comprehensive knowledge about the institution, rely on specific information that is sometimes outdated, require additional systems for information retrieval, and experience high turnover rates. This leads to associated costs and issues related with outdated information, resulting in inaccurate responses and long waiting times. Generative artificial intelligence models, known as Large Language Models (LLMs), offer an interesting alternative for an automated virtual help desk. These models can understand even vague and poorly structured questions and generate reasonably appropriate answers. However, they are not without flaws, as they tend to present issues like "hallucinations" when the required information is not present in their training data. To minimize this problem, it is crucial to ensure that the model has precise and comprehensive information, which needs a specific methodology for information collection, validation, and updating. Base models require an adaptation process to be used for specific cases, for which techniques like Fine-Tuning and Retrieval Augmented Generation (RAG) exist. Fine-tuning retrains a model’s weights with new specific information, while RAG uses both proprietary information—in this case, from the university—and publicly available internet data. Both techniques have pros and cons that need to be evaluated to select the most suitable option. They also demand appropriate and specialized infrastructure, which is often expensive. Thus, another challenge is to find a balance between suitable equipment and reasonable costs. The final system, from the user’s perspective, must be accurate, flexible, and adaptable to deliver a satisfactory experience. As the results show, Aurel_AI represents an advance in the digitalization of educational services, standing out for its ability to generate accurate and personalized responses. However, its current limitations, such as handling concurrent queries and hallucinations, underscore the need for adjustments to both infrastructure and data processing methodology. With strategic improvements, the system has the potential to consolidate itself as a replicable model for multiple university digital services.

Full Text