Big Data: Fundamental Questions

Big Data Week is the second theme of Software Engineering Daily.

“Big Data” is a legitimate trend as well as an overhyped buzzword. Big Data is not a scam, but putting “Big Data” on your resume today has negative expected value.

There is tension and confusion around the reality and surreality of Big Data engineering.

From August 3-9, there will be podcast interviews with experts on Spark, Hadoop, Zookeeper, Kafka, and other components of the Big Data ecosystem. Below is a collection of fundamental questions about the space:

  • How do customers of multiple managed big data companies deal with the heterogeneity?
  • Are there enough knowledgeable support technicians at managed big data companies to handle the customers?
  • How does a big data customer augment a batch pipeline with streaming?
  • Are Hadoop queries mostly written in Pig or Hive?
  • Is a measurement of Big Data throughput the new Moore’s Law?
  • Where does Kafka fit in?
  • What is the difference between Spark, Storm, Flink, Samza, and other streaming technologies?
  • How has the database world been affected by Big Data?
  • What is at the intersection of JavaScript and Big Data?
  • What is at the intersection of Bitcoin and Big Data?

Software Daily

Software Daily

Subscribe to Software Daily, a curated newsletter featuring the best and newest from the software engineering community.