Deterministic Model for Distributed Speculative Stream Processing

I. Kuralenok; Trofimov A.; Marshalkin N.; B. Novikov

doi:10.1007/978-3-319-98398-1\_16

Publications

?

Deterministic Model for Distributed Speculative Stream Processing

P. 233–246.

Kuralenok I., Trofimov A., Marshalkin N., Novikov B.

Users of modern distributed stream processing systems have to choose between non-deterministic computations and high latency due to a need in excessive buffering. We introduce a speculative model based on MapReduce-complete set of operations that allows us to achieve determinism and low-latency. Experiments show that our prototype can outperform existing solutions due to low overhead of optimistic synchronization.

Keywords: Data streams Distributed processing Drifting state

In book

Lecture Notes in Computer Science

Vol. 11019: Advances in Databases and Information Systems . , Springer, 2018.

Scalability and Parallelization of Sequential Processing: Big Data Demands and Information Algebras

Golubtsov P., , in: Advances in Intelligent Systems and Computing book series Vol. 1127. Advances in Intelligent Systems, Computer Science and Digital EconomicsVol. 1127: Advances in Intelligent Systems, Computer Science and Digital Economics.: Switzerland: Springer, 2020. P. 274–298.

Procedures of sequential updating of information are important for “big data streams” processing because they avoid accumulating and storing large data sets. As a model of information accumulation, we study the Bayesian updating procedure for linear experiments. Analysis and gradual transformation of the original processing scheme in order to increase its efficiency lead to certain ...

Added: March 16, 2020

FlameStream: Model and Runtime for Distributed Stream Processing

Kuralenok I., Trofimov A., Marshalkin N. et al., , in: Proceedings of the 5th ACM SIGMOD Workshop on Algorithms and Systems for MapReduce and Beyond.: ACM, 2018. P. 8:1–8:2.

Exactly-once semantics without high latency overhead is still hard to achieve within state-of-the-art stream processing systems.We introduce a model providing for exactly-once using lightweight optimistic approach for obtaining determinism and idempotence. We show its feasibility with a prototype. ...

Added: February 13, 2019