Files and links (2)
Published (Version of record)CC BY V4.0, Open Access
Published (Version of record)CC BY V4.0, Open
Conference proceeding
CAShift: Benchmarking Log-Based Cloud Attack Detection under Normality Shift
Proceedings of the ACM on software engineering, Vol.2(FSE), pp.1687-1709
19/06/2025
Metrics
1 Record Views
Abstract
With the rapid advancement of cloud-native computing, securing cloud environments has become an important task. Log-based Anomaly Detection (LAD) is the most representative technique used in different systems for attack detection and safety guarantee, where multiple LAD methods and relevant datasets have been proposed. However, even though some of these datasets are specifically prepared for cloud systems, they only cover limited cloud behaviors and lack information from a whole-system perspective. Another critical issue to consider is normality shift, which implies that the test distribution could differ from the training distribution and highly affect the performance of LAD. Unfortunately, existing works only focus on simple shift types such as chronological changes, while other cloud-specific shift types are ignored, e.g., different deployed cloud architectures. Therefore, a dataset that captures diverse cloud system behaviors and various types of normality shifts is essential.
To fill this gap, we construct a dataset CAShift to evaluate the performance of LAD in cloud, which considers different roles of software in cloud systems, supports three real-world normality shift types (application shift, version shift, and cloud architecture shift), and features 20 different attack scenarios in various cloud system components. Based on CAShift, we conduct a comprehensive empirical study to investigate the effectiveness of existing LAD methods in normality shift scenarios. Additionally, to explore the feasibility of shift adaptation, we further investigate three continuous learning approaches, which are the most common methods to mitigate the impact of distribution shift. Results demonstrated that 1) all LAD methods suffer from normality shift where the performance drops up to 34%, and 2) existing continuous learning methods are promising to address shift drawbacks, but the ratio of data used for model retraining and the selection of algorithms highly affect the shift adaptation, with an increase in the F1-Score of up to 27%. Based on our findings, we offer valuable implications for future research in designing more robust LAD models and methods for LAD shift adaptation.
Details
- Title
- CAShift: Benchmarking Log-Based Cloud Attack Detection under Normality Shift
- Creators
- Jiongchi Yu - Singapore Management UniversityXiaofei Xie - Singapore Management UniversityQiang Hu - Tianjin UniversityBowen Zhang - Singapore Management UniversityZiming Zhao - Zhejiang UniversityYun Lin - Shanghai Jiao Tong UniversityLei Ma - The University of TokyoRuitao Feng - Singapore Management UniversityFrank Liauw - Government Technology Agency of Singapore (Singapore)
- Publication Details
- Proceedings of the ACM on software engineering, Vol.2(FSE), pp.1687-1709
- Publisher
- ACM; New York, USA
- Number of pages
- 23
- Grant note
- This research is partially supported by the Lee Kong Chian Fellowship, the National Research Foundation, Singapore, and the Cyber Security Agency under its National Cybersecurity R&D Programme (NCRP25-P04-TAICeN).
- Identifiers
- 991013294052202368
- Copyright
- © 2025 Copyright held by the owner/author(s).
- Academic Unit
- Faculty of Science and Engineering
- Language
- English
- Resource Type
- Conference proceeding