Conference proceeding
AUCAD: Automated Construction of Alignment Dataset from Log-Related Issues for Enhancing LLM-based Log Generation
Proceedings of the 16th International Conference on Internetwar(2025)e, pp.413-425
ACM Other Conferences
Internetware 2025: the 16th International Conference on Internetware, 16th (Trondheim, Norway, 20/06/2025–22/06/2025)
27/10/2025
Metrics
16 Record Views
Abstract
Log statements have become an integral part of modern software systems. Prior research efforts have focused on supporting the decisions of placing log statements, such as where/what to log. With the increasing adoption of Large Language Models (LLMs) for code-related tasks such as code completion or generation, automated approaches for generating log statements have gained much momentum. However, the performance of these approaches still has a long way to go. This paper explores enhancing the performance of LLM-based solutions for automated log statement generation by post-training LLMs with a purpose-built dataset. Thus the primary contribution is a novel approach called AUCAD, which automatically constructs such a dataset with information extracting from log-related issues. Researchers have long noticed that a significant portion of the issues in the open-source community are related to log statements. However, distilling this portion of data requires manual efforts, which is labor-intensive and costly, rendering it impractical. Utilizing our approach, we automatically extract log-related issues from 1,537 entries of log data across 88 projects and identify 808 code snippets (i.e., methods) with retrievable source code both before and after modification of each issue (including log statements) to construct a dataset. Each entry in the dataset consists of a data pair representing high-quality and problematic log statements, respectively. With this dataset, we proceed to post-train multiple LLMs (primarily from the Llama series) for automated log statement generation. Both human and experimental evaluations indicate that these models significantly outperform existing LLM-based solutions, thereby validating the efficacy of our method for constructing a post-training dataset to enhance LLM-based log statement generation.
Details
- Title
- AUCAD: Automated Construction of Alignment Dataset from Log-Related Issues for Enhancing LLM-based Log Generation
- Creators
- Hao Zhang - Nanjing UniversityDongjun Yu - Nanjing UniversityLei Zhang - Nanjing UniversityGuoping Rong - Nanjing UniversityYongda Yu - Nanjing UniversityHaifeng Shen - Southern Cross UniversityHe Zhang - Nanjing UniversityDong Shao - Nanjing UniversityHongyu Kuang - Nanjing University
- Contributors
- Hong Mei (Editor) - Peking UniversityJian Lv (Editor)Zhi Jin (Editor) - Peking UniversityXuandong Li (Editor) - Nanjing UniversityThomas Zimmermann (Editor)Ge Li (Editor) - Peking UniversityLei Bu (Editor) - Nanjing UniversityXin Xia (Editor)
- Publication Details
- Proceedings of the 16th International Conference on Internetwar(2025)e, pp.413-425
- Conference
- Internetware 2025: the 16th International Conference on Internetware, 16th (Trondheim, Norway, 20/06/2025–22/06/2025)
- Series
- ACM Other Conferences
- Publisher
- ACM
- Number of pages
- 13
- Identifiers
- 991013325411002368
- Copyright
- © 2025 Copyright held by the owner/author(s).
- Academic Unit
- Faculty of Science and Engineering
- Language
- English
- Resource Type
- Conference proceeding