Dataflow programming is a fundamental software paradigm widely adopted in various applications, ranging from Signal Processing to Artificial Intelligence (AI), Autonomous Vehicles (AV), Data Processing and Robotics. The underlying concept is rather simple: a computing task is decomposed into a graph of operators through which flows the data to be processed. For instance, a collision detection system in an AV can be modeled as the chaining of (i) a video stream that captures the surroundings, (ii) an object detection algorithm that, for each frame, identifies the obstacles and (iii) a prediction algorithm that, based on the identified entities, anticipates collisions.
Traditionally, these applications run in bounded and homogeneous execution environments where all computational, networking, and storage resources reside at predefined static locations. However, edge computing is now spreading those resources along a cloud-to-thing continuum that is capable of hosting applications anywhere, from big cloud servers to small single board computers and microcontrollers. This ubiquitous runtime is an opportunity for AI, AV, and Robotics applications as it can augment their computing and resilience capabilities. At the same time, existing dataflow programming frameworks are tailored to work either in cloud environments with data analytics workloads or in embedded systems for real-time data processing. As a consequence, these frameworks struggle to cope with the great dynamicity and heterogeneity of the edge computing infrastructure and more importantly are unable to cover the needs of applications deployed along the clout-to-thing continuum.
In this presentation we will first go through the current landscape of dataflow programming frameworks and the corresponding novel requirements arising from the recent advancements in AI, AV, and Robotics. This will serve as the foundation for the design of a novel dataflow programming framework that natively accounts for the characteristics of edge computing and forthcoming applications, including decentralization, distribution, mobility, and resilience.
We will then present the model and primitives of our novel dataflow framework that is built on the technologies developed in the Eclipse Edge Native Working Group. As a result, developers are empowered with a simple API for writing end-to-end applications along the cloud-to-thing continuum. Finally, we will demonstrate an exemplary end-to-end computer vision application that leverages our novel dataflow framework.