Big Data and analytics are going to change in this COVID-19 era. I want to share with everyone the same messages that I’m giving my own clients. I’m hoping that this post will help those data teams that aren’t living up to their potential to start...
Update: Confluent has renamed KSQL to ksqlDB It’s that Kafka Summit time of year again. There are lots of announcements. Some are good and some you have to sift through in order to figure out what’s the best for you and your organization. I’ll...
Note: This post was guest written by John Desmond. My preparation for the course began before I knew about the course, and before I realized that I wanted to specialize in data engineering. When I decided I wanted to learn programming, I hadn’t quite decided if...
At DataEngConf Barcelona, I premiered a new talk about the importance of creating a data engineering culture. I share what a data engineering culture is and what management needs to do to be successful with Big Data. You can download the slides from the talk here and...
There is a common misunderstanding in data engineering that you can do everything you need to create a Big Data data pipeline with SQL. This notion is being promoted by some vendors and companies. They’re wrong and you can’t do all of your data engineering...
A common use case for using Kafka and Pulsar is to create work queues. The two technologies offer different implementations for accomplishing this use case. I’ll discuss the ways of implementing work queues in Kafka and Pulsar as well as the relative strengths...
