Getting a system to work and getting it to work reliably enough to be a production process is a problem many teams have yet to crack according to Gartner who say 80% of Hadoop systems are still not in production. And Hadoop is just one workload of many that should be supported.
Based on my experience working with a large number of teams who have had big data systems in production for years, I will describe the patterns and practices of highly successful DataOps teams that get things right. This will include some technical patterns and some social patterns with an emphasis on actionable advice for building systems that matter.
//
Ted Dunning
@ted_dunning
is Chief Application Architect at MapR. Ted has worked in a number of successful startups that have been pioneers in working with data. Teams he has led produced one of the first major music recommendation engines, advances in streaming media, advanced behavioral fraud detection systems and more. He has also been very active in open source for decades contributing to a number of projects and mentoring many more, and currently serves on the board of directors of the Apache Software Foundation.