Stream processing is becoming mainstream. It is present in thousands of companies around the globe and used in a variety of applications including real-time monitoring, data analytics, car-trip pricing, credit card fraud detection, and many others. Despite the great success of the stream processing paradigm, testing a networked stream processing application (particularly at scale) is still a cumbersome and often expensive process. Existing approaches typically include building an expensive testbed, applying for a slot on a community one (e.g., GENI, CloudLab) or hiring a cloud-based set-up. On top of that, developers still need to spend significant amounts of time configuring and managing their platforms, e.g., by connecting cluster nodes and ingesting data. In this project, we aim at lowering the wall (from both time and money perspective) for stakeholders to participate in the stream processing ecosystem. For that, we are building a simple and easy-to-use prototyping tool for distributed stream processing applications. It is a whole stream processing setup in a single server and in a matter of minutes. We envision this work can also help paving the way for more reproducible research in the stream processing domain, a basic need for the research community.
Are data streaming platforms ready for a mission critical world? Md. Monzurul Amin Ifath, Miguel Neves, Brandon Bremner, Jeff White, Tomas Szeredi, Israat Haque Submitted at IEEE Communications Magagine 2024
Fast Prototyping of Distributed Stream Processing Applications with stream2gym Md. Monzurul Amin Ifath, Miguel Neves, Israat Haque IEEE ICDCS 2023
Raptor: Rapid prototyping of distributed stream processing applications at scale Md. Monzurul Amin Ifath, Miguel Neves, Israat Haque ACM CoNEXT Posters and Demos 2021
israat at dal dot ca
monzurul dot amin at dal dot ca
em244217 at dal dot ca
mg478789 at dal dot ca