Generative AI-based Approach to Concept Drift Generation in Streaming Text Data

WSEAS Transactions on Information Science and Applications. 2025. Vol. 22. P. 11–20.

Belov B., Peter Panfilov

Real-time analysis of text streams is crucial for industrial and business processes and scenarios. It is expected to be  one of the important future research topics in the text processing and understanding domain. Analysis of text data is based on the use of pre-trained machine learning/data mining (ML/DM) models that may demonstrate performance degradation over time due to the drift in text data. The problem of tracking drift in data and quickly retraining a model in response to changes in the operational environment represents a great challenge in product model environments. We discuss and evaluate an approach to artificially generating concept drift aimed at providing test data for valuating model performance and improving its accuracy. Existing methods for generating concept drift in text streams are limited to specific domains and are not universally applicable. This paper explores approaches for generating concept drift in text streams using the latest developments in generative artificial intelligence (GenAI) such as Large Language Models (LLMs). Two methods for generating concept drift with LLMs are proposed and compared to existing techniques. The comparison demonstrates that concept drift generation using LLMs is more effective than traditional methods. Additionally, LLMs can rapidly produce complex concept drift scenarios that are significantly more challenging to generate with standard approaches.

Keywords: concept drift, large language models, generative AI, lifelong machine learning, text data streams, data drift generation, model retraining, drift generation methods