Kafka-Streams with Spring-Boot | Part1

  • The first one is known as the input/output channel binding. The input/output channel binding defines the list of sources and the destinations. We are defining only one channel. The name of the channel is “input-channel-1”, and the destination is the “users”. This means that, In this configuration, We shall connect to “users” Kafka topic using the “input-channel-1”. Because we wanted to connect to the “users” Kafka topic and read all the messages.
  • The second one is known as the binder. The binder will define our source and destination technology.
  • Apache Kafka Binder → It implements the Kafka Client APIs.
  • Kafka Streams Binder → It is explicitly designed for Kafka Streams API.
  • This method will read from a Kafka topic and return a KStream. The KStream is a Kafka message stream made up of a string key and a String value, here in our case.
  • We also need to annotate this method using @Input annotation and pass in the input channel name. That’s all.
  • We want to bind this class with the Spring-Cloud-Stream infrastructure and for the same purpose, we are using @EnableBinding annotation. Into this, we pass in our “binder-interface-name”. This class will trigger the Spring-Cloud-Stream framework and connect to the Kafka input channel using the Kafka Streams API and start consuming the input messages as a KStream.
  • The next step is to process each message from the KStream. This method is @StreamListener for the given input channel. The Spring Cloud framework will call this method and pass in the KStream. Now, we can do whatever we want to do with this KStream.
  • In this use-case, we would simply log the key-value-message thus being received. So, we shall use the foreach() method on the input stream which takes in key/value lambda, and log the key and the value.




Software Engineer for Big Data distributed systems

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Microsoft Excel For Mac Squared Symbol

Mongo Spring Multiple Server Config setting

Transforming my ordinary laptop to a deep learning research laboratory in 12 steps

The Functional Sandwich Principal

Building a Google Clone — Part 4: Deploying to Firebase

Code on a laptop

So, You Suck at Coding? Embrace It!

Creating a Simple OpenAPI Spec

Another unowned/weak article

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
aditya goel

aditya goel

Software Engineer for Big Data distributed systems

More from Medium

Decouple Configuration from Application Code with Spring Cloud Config

Elasticsearch REST Java Client

Kafka — Connect to Secure Cluster

Distributed Tracing with Spring Cloud Sleuth