前言

kafka消息中间键也是我们经常要用到的功能，尤其是在大数据、高并发的项目中，如日志收集、业务数据分发等等。其最核心的俩大功能作用是:①削峰填谷②异步解耦。本节我们主要介绍一下如何在springboot项目中集成kafka消息中间键，实现简单的数据分发以及消费的案例。

正文

• kafka集群搭建

快速搭建一个kafka集群，我们这里以docker环境为例，搭建一个kafka集群，具体搭建过程请参考以下博客，这里不再赘述。

（六）史上最强ELK集群搭建系列教程——kafka集群搭建_北溟的博客-CSDN博客

• springboot项目引入kafka依赖

<dependency>
	<groupId>org.springframework.kafka</groupId>
	<artifactId>spring-kafka</artifactId>
</dependency>

• 引入kafka配置

spring:
  #kafka配置
  kafka:
    #kafka集群地址
    bootstrap-servers: 192.168.56.10:9091,192.168.56.10:9092,192.168.56.10:9093
    producer:
      #批量发送的数据量大小
      batch-size: 16384
      #可用发送数量的最大缓存
      buffer-memory: 33554432
      #key序列化器
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      #value序列化器
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
      properties:
        linger.ms: 1
    consumer:
      enable-auto-commit: false
      auto-commit-interval: 100ms
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      properties:
        session.timeout.ms: 15000
      group-id: atp-group-id

• 创建一个kafka生产者生产消息

①创建一个kafka生产者类KafkaProducer生产消息

package com.yundi.atp.platform.kafka;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.stereotype.Component;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.util.concurrent.ListenableFutureCallback;

/**
 * @Author: yanp
 * @Description: kafka生产者
 * @Date: 2021/11/3 18:31
 * @Version: 1.0.0
 */
@Component
@Slf4j
public class KafkaProducer 
    @Autowired
    private KafkaTemplate<Integer, String> kafkaTemplate;

    public void sendMsg(String topic, String msg) 
        log.info("开始发送kafka消息:[topic:,msg:]", topic, msg);
        ListenableFuture<SendResult<Integer, String>> sendMsg = kafkaTemplate.send(topic, msg);
        //消息确认
        sendMsg.addCallback(new ListenableFutureCallback<SendResult<Integer, String>>() 
            @Override
            public void onFailure(Throwable throwable) 
                log.error("send error:[ex:,topic:,msg:]", throwable, topic, msg);
            

            @Override
            public void onSuccess(SendResult<Integer, String> stringStringSendResult) 
                log.info("send success:[topic:,msg:]", topic, msg);
            
        );
    

 ②通过web服务发送生产消息测试

package com.yundi.atp.platform.module.test.controller;

import com.yundi.atp.platform.common.Result;
import com.yundi.atp.platform.kafka.KafkaProducer;
import io.swagger.annotations.Api;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * @Author: yanp
 * @Description:
 * @Date: 2021/11/4 17:32
 * @Version: 1.0.0
 */
@Api(tags = "Springboot集成Kafka测试")
@RestController
@RequestMapping(value = "/kafka")
public class KafkaContoller 
    @Autowired
    private KafkaProducer kafkaProducer;

    @GetMapping(value = "/sendKafkaTestMsg/topic/msg")
    public Result sendKafkaTestMsg(@PathVariable(value = "topic")String topic,@PathVariable(value = "msg")String msg)
        kafkaProducer.sendMsg(topic,msg);
        return Result.success();
    

③通过测试页面发送生产者消息

<template>
  <div class="container">
    <div class="title">
      <span>Springboot集成kafka案例</span>
      <el-divider direction="vertical"></el-divider>
      <router-link to="home">
        <span style="font-size: 18px;">退出</span>
      </router-link>
    </div>
    <el-divider>Test Staring</el-divider>
    <div style="width: 400px;background: #ddd;padding: 40px 20px;">
      <el-form ref="form" :model="form" label-width="70px" class="login">
        <el-form-item label="发送主题" prop="topic">
          <el-input v-model="form.topic"></el-input>
        </el-form-item>
        <el-form-item label="发送消息" prop="name">
          <el-input v-model="form.msg"></el-input>
        </el-form-item>
        <el-button type="primary" @click="sengMsg" style="width: 100%;margin: 0;">立即发送</el-button>
      </el-form>
    </div>
  </div>
</template>

<script>
export default 
  name: "Kafka",
  data() 
    return 
      form: 
        topic: 'atp',
        msg: 'hello world!',
      
    
  ,
  methods: 
    sengMsg() 
      this.$http.get('/kafka/sendKafkaTestMsg/'+this.form.topic+"/"+this.form.msg,).then(res => 
        if (res.data.code === 1) 
          this.$refs.form.resetFields();
          this.$message.success(res.data.msg);
         else 
          this.$refs.form.resetFields();
          this.$message.warning(res.data.msg);
        
      ).catch(error => 
        this.$message.error(error);
      );
    
  

</script>

<style scoped lang="scss">
.container 
  padding: 10px;

</style>

 

 

• 创建kafka消费者消费消息

package com.yundi.atp.platform.kafka;

import lombok.extern.slf4j.Slf4j;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

/**
 * @Author: yanp
 * @Description: kafka消费者
 * @Date: 2021/11/3 18:32
 * @Version: 1.0.0
 */
@Component
@Slf4j
public class KafkaConsumer 

    /**
     * 消费者：监听test主题，分组：groupId1
     */
    @KafkaListener(topics = "test", groupId = "groupId1")
    public void consumer1(ConsumerRecord<Integer, String> record) 
        log.info("consumer1 start consume message:[topic:,msg:]", record.topic(), record.value());
    

    /**
     * 消费者：监听atp主题，分组：groupId1
     */
    @KafkaListener(topics = "atp", groupId = "groupId1")
    public void consumer2(ConsumerRecord<Integer, String> record) 
        log.info("consumer2 start consume message:[topic:,msg:]", record.topic(), record.value());
    

    /**
     * 消费者：监听atp主题，分组：groupId3
     */
    @KafkaListener(topics = "atp", groupId = "groupId2")
    public void consumer3(ConsumerRecord<Integer, String> record) 
        log.info("consumer3 start consume message:[topic:,msg:]", record.topic(), record.value());
    

• 结果测试

①发送生产者消息

②消费者消费消息

结语

ok,到这里关于springboot集成kafka案例实战就结束了，我们下期见。。。