Apache Spark Streaming

Note: Spark Streaming is the previous generation of Spark’s streaming
engine. There are no longer updates to Spark Streaming and it’s a legacy project. There is a newer and easier to use streaming engine in Spark called Structured Streaming.

Spark Structured Streaming is a scalable fault-tolerant streaming processing system that natively supports streaming workloads. Streaming computation therefore could be expressed the same way a batch computation on static data would be expressed. The Spark SQL engine will take care of running it not only incrementally but continuously, and updating the final result as streaming data continues to arrive. Spark streaming also accepts many sources, programming languages like Scala, Java, Python, or R are adoptable to perform tasks like time-based windows, aggregations, joining streaming and static data, etc. These tasks are executed using the same optimized Spark SQL engine.