Kafka-Streams with Spring-Boot | Part1

  • The first one is known as the input/output channel binding. The input/output channel binding defines the list of sources and the destinations. We are defining only one channel. The name of the channel is “input-channel-1”, and the destination is the “users”. This means that, In this configuration, We shall connect to “users” Kafka topic using the “input-channel-1”. Because we wanted to connect to the “users” Kafka topic and read all the messages.
  • The second one is known as the binder. The binder will define our source and destination technology.
  • Apache Kafka Binder → It implements the Kafka Client APIs.
  • Kafka Streams Binder → It is explicitly designed for Kafka Streams API.
  • This method will read from a Kafka topic and return a KStream. The KStream is a Kafka message stream made up of a string key and a String value, here in our case.
  • We also need to annotate this method using @Input annotation and pass in the input channel name. That’s all.
  • We want to bind this class with the Spring-Cloud-Stream infrastructure and for the same purpose, we are using @EnableBinding annotation. Into this, we pass in our “binder-interface-name”. This class will trigger the Spring-Cloud-Stream framework and connect to the Kafka input channel using the Kafka Streams API and start consuming the input messages as a KStream.
  • The next step is to process each message from the KStream. This method is @StreamListener for the given input channel. The Spring Cloud framework will call this method and pass in the KStream. Now, we can do whatever we want to do with this KStream.
  • In this use-case, we would simply log the key-value-message thus being received. So, we shall use the foreach() method on the input stream which takes in key/value lambda, and log the key and the value.

--

--

--

Software Engineer for Big Data distributed systems

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Google Cloud: Managed File Transfer using Storage Transfer Service

BDD Addict Newsletter November-December 2019

More Productive Editor Layout in Unity

Formatting and Automation

Bigturn Dynamic Application Form

Your first app in Flutter can be an app with 10k active users

Introduction To Low-Level Design

Understanding How the Chrome V8 Engine Translates JavaScript into Machine Code

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
aditya goel

aditya goel

Software Engineer for Big Data distributed systems

More from Medium

Apache Kafka: Core Concepts and Use Cases

AWS SERVERLESS ARCHITECTURE IN JAVA

Spring boot Elasticsearch Remote Access

Getting Started with Apache Kafka and Spring Boot