Journal article
Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review
ACM transactions on software engineering and methodology, Vol.34(1), pp.1-26
31/01/2025
Metrics
12 File views/ downloads
31 Record Views
Abstract
As code review is a tedious and costly software quality practice, researchers have proposed several machine learning-based methods to automate the process. The primary focus has been on accuracy, that is, how accurately the algorithms are able to detect issues in the code under review. However, human intervention still remains inevitable since results produced by automated code review are not 100% correct. To assist human reviewers in making their final decisions on automatically generated review comments, the comprehensibility of the comments underpinned by accurate localization and relevant explanations for the detected issues with repair suggestions is paramount. However, this has largely been neglected in the existing research. Large language models (LLMs) have the potential to generate code review comments that are more readable and comprehensible by humans thanks to their remarkable processing and reasoning capabilities. However, even mainstream LLMs perform poorly in detecting the presence of code issues because they have not been specifically trained for this binary classification task required in code review. In this paper, we contribute Carllm (Comprehensibility of Automated Code Review using Large Language Models), a novel fine-tuned LLM that has the ability to improve not only the accuracy but, more importantly, the comprehensibility of automated code review, as compared to state-of-the-art pre-trained models and general LLMs.
Details
- Title
- Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review
- Creators
- Yongda Yu (Researcher) - Nanjing UniversityGuoping Rong (Corresponding Author) - Nanjing UniversityHaifeng Shen (Contributor) - Southern Cross University, Faculty of Science and EngineeringHe Zhang (Contributor) - Nanjing UniversityDong Shao (Contributor) - Nanjing UniversityMin Wang (Contributor) - Tencent Technology Cooperation Ltd.Zhao Wei (Contributor) - Tencent Technology Cooperation Ltd.Yong Xu (Contributor) - Tencent Technology Cooperation Ltd.Juhong Wang (Contributor) - Tencent Technology Cooperation Ltd.
- Publication Details
- ACM transactions on software engineering and methodology, Vol.34(1), pp.1-26
- Publisher
- Association for Computing Machinery
- Number of pages
- 26
- Grant note
- Copyright © 2024 Copyright held by the owner/author(s).
- Identifiers
- 991013222313202368
- Academic Unit
- Faculty of Science and Engineering
- Language
- English
- Resource Type
- Journal article