Stream Processing with Satish Mittal
“We still need to see in the long run how much of community and industry adoption is there. Because at the end of the day, these are the single two most important things which define and determine the success of any platform.”
Real-time stream processing of data is becoming widely adopted in the efforts to manage and process “big data”. Some of the top frameworks for processing data streams include Storm, Spark, Samza and Flink.
Satish Mittal is an architect at InMobi, an ad platform that needs to deal with processing large volumes of ad data with tight time demands. He recently conducted an analysis of streaming frameworks to determine which of the tools was right for the application at InMobi.
Questions
- What is real-time stream processing?
- What is micro-batching, and what are the problems with it?
- What were the streaming platforms you were trying to decide between at InMobi?
- Did you look at the Lambda architecture when you were building your system?
- How do Spark and Storm compare when it comes to message delivery guarantees and fault tolerance?
- What platform did you finally end up choosing?
- What do you see as the future of the big data stack?
