IEEE Big Data 2018 Congress in San Francisco, CA
IEEE Big Data 2018 Congress in San Francisco, CA
Several hundreds of researchers and practitioners descended to San Francisco, CA to discuss all aspects of Big Data and more generally services. Giselle van Dongen (UGent PhD candidate & Data Scientist at Klarrio - @Klarr_io on Twitter) presented her work-in-progress paper “Latency Measurement of Fine-Grained Operations in Benchmarking Distributed Stream Processing Frameworks” (co-authors: Bram Steurtewagen and Dirk Van den Poel). This paper describes a benchmark for streaming frameworks to allow for accurate latency benchmarking of fine-grained individual stages of a processing pipeline. By determining the latency of distinct common operations in the processing flow instead of the end-to-end latency, we can form guidelines for efficient processing pipeline design. In this work, we use Apache Flink to validate the benchmark setup. The full paper will be published later this year. We thank Klarrio - The Lingua Franca Advocates for IoT and Analytics - for their continued (financial and other) support!
Friday, 6 July 2018