TECHNIQUES IN REAL-TIME BIG DATA ANALYTICS: AN APPRAISAL OF ADVANTAGES, LIMITATIONS, AND APPLICATIONS

Abstract

The rapid growth of connected devices, social media, and digital services has produced unprecedented data volumes, driving a shift from batch processing to real-time analytics (RTA). RTA emphasizes velocity, enabling data to be ingested, processed, and analyzed as it arrives to deliver immediate insights and business value. This article reviews major RTA techniques, including stream processing (SP), complex event processing (CEP), query- and rule-based analytics, programmatic platforms, anytime algorithms, and hybrid edge–cloud architectures. The paper assesses stream processing frameworks such as Apache Flink, Kafka, and Storm for scalability, fault tolerance, and latency, with Flink consistently achieving sub-100-millisecond performance for complex tasks. CEP is highlighted for detecting event patterns and correlations, supporting applications in fraud detection, predictive maintenance, and smart environments. Anytime algorithms are examined for their ability to provide progressively refined results, enabling rapid decision-making in time-sensitive contexts. The integration of machine learning into streaming pipelines, edge deployment strategies guided by the latency–bandwidth–compute triangle, and emerging hybrid architectures are also discussed. Three developmental trends are identified: evolution from batch to stream-native systems, convergence of batch and stream paradigms, and fusion of machine learning with RTA. Findings underscore the need to balance latency, cost, and complexity when selecting techniques for specific use cases.