In my book Data Engineering Teams, I talk about a skill that’s often overlooked and unknown to data engineering teams. Teams often don’t know they need a veteran, think they can’t afford a veteran, or don’t understand why you need a veteran on...
In my seminal post On Complexity in Big Data I talked about the level of complexity increase with Big Data. The post itself focused on Big Data batch systems. I didn’t really cover real-time complexity increases when dealing with Big Data. In the post, I argue...
I’m often asked how someone who is a consultant how they can get into Big Data. This is an important subject because it will define your success as consultant in the field. More importantly, it will define how successful your customers will be. Learning If Big...
Today’s blog post comes from a question from a subscriber on my mailing list. The questions come from Vaughn S.: How is programming used in data engineering? What do I have to offer at meetups? How can I round out my skillset? How is programming used in data...
Writing your own distributed system shouldn’t be a task you undertake lightly. Too often, I’m seeing teams create their own distributed system. In my experience, this is because they don’t know or think about all of the ramifications of creating...
