I'm giving a talk at UC Santa Barbara

I’m giving a talk at the UC Santa Barbara CS Colloquium.

Rethinking Streaming Dataflow Systems: A Case for Disaggregated, Adaptive, and Resource-Aware Engines

Abstract: Streaming dataflow systems, like Apache Flink, have recently gained widespread adoption in industry and academia, while all major cloud providers offer data stream analytics as managed services. However, the increasing diversity of streaming applications and the availability of compute-enabled edge devices, hardware accelerators, and new cloud computing paradigms, is pushing existing dataflow systems to their limits. In this talk, I will describe our ongoing research towards addressing three major shortcomings of the streaming platforms status quo: (i) rigid, monolithic designs, inherited from MapReduce-style predecessors, (ii) homogeneous resource assumptions that hinder the deployment of emerging use cases, like machine learning pipelines and edge analytics, and (iii) notoriously complex to configure infrastructure. I will present our recent results on adaptive reconfiguration and resource-conscious optimizations for stream processing, and I will share our vision towards developing a novel fully-disaggregated streaming runtime architecture.