반응형
/*
-- Title : [Kfk3.2] Kafka 설치-2(Kafka 구성 및 서버 구동)
-- Tag : 아파치 카프카 apache kafka
*/
■ Kafka 설치 위한 선행 처리
■ Kafka 다운로드 및 압축 해제
[kafka@ka01 bin]$ cd /tmp
[kafka@ka01 tmp]$ wget https://mirror.navercorp.com/apache/kafka/3.2.3/kafka_2.12-3.2.3.tgz
--2022-09-22 10:38:52-- https://mirror.navercorp.com/apache/kafka/3.2.3/kafka_2.12-3.2.3.tgz
Resolving mirror.navercorp.com (mirror.navercorp.com)... 125.209.216.167
Connecting to mirror.navercorp.com (mirror.navercorp.com)|125.209.216.167|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104120289 (99M) [application/octet-stream]
Saving to: ‘kafka_2.12-3.2.3.tgz’
100%[============================================================>] 104,120,289 6.56MB/s in 16s
2022-09-22 10:39:08 (6.03 MB/s) - ‘kafka_2.12-3.2.3.tgz’ saved [104120289/104120289]
[kafka@ka01 tmp]$ ll kafka*
-rw-rw-r--. 1 kafka kafka 104120289 9월 17 15:42 kafka_2.12-3.2.3.tgz
kafka@ka01 tmp]$ sudo tar zxf kafka_2.12-3.2.3.tgz -C /opt
[sudo] kafka의 암호:
[kafka@ka01 opt]$ cd /opt
[kafka@ka01 opt]$ sudo ln -s kafka_2.12-3.2.3.tgz kafka
[sudo] kafka의 암호:
[kafka@ka01 opt]$ ll kafka
lrwxrwxrwx. 1 root root 20 9월 22 11:38 kafka -> kafka_2.12-3.2.3.tgz
[kafka@ka01 opt]$ sudo mkdir data
[sudo] kafka의 암호:
[kafka@ka01 opt]$ ll
합계 0
drwxr-xr-x. 8 root root 136 9월 12 21:34 VBoxGuestAdditions-6.1.36
drwxr-xr-x. 7 root root 145 9월 22 09:50 apache-zookeeper-3.6.3-bin
drwxr-xr-x. 2 root root 6 9월 22 13:17 data
lrwxrwxrwx. 1 root root 16 9월 22 12:59 kafka -> kafka_2.12-3.2.3
drwxr-xr-x. 7 root root 133 9월 22 13:02 kafka_2.12-3.2.3
drwxr-xr-x. 2 root root 6 10월 31 2018 rh
lrwxrwxrwx. 1 root root 26 9월 22 09:45 zookeeper -> apache-zookeeper-3.6.3-bin
■ Kafka 설정 파일 수정
[kafka@ka01 opt]$ sudo vi kafka/config/server.properties
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
...
# ka01 서버 설정(서버마다 1,2,3 달리 설정)
broker.id = 1
# 메시지 데이터 저장 기본 디렉토리 설정
log.dirs=/opt/data
# Zookeeper 포트 설정(모든 서버 기록)
# ka01, ka02, ka03 대신 IP 등록 가능
zookeeper.connect=ka01:2181,ka02:2181,ka03:2181
■ Zookeeper 및 Kafka 구동
ㅁ Zookeeper 구동
## Zookeeper 상태 확인
[kafka@ka03 bin]$ sudo /opt/zookeeper/bin/zkServer.sh start
[sudo] kafka의 암호:
/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
## Zookeeper 상태 확인
[kafka@ka01 opt]$ sudo /opt/zookeeper/bin/zkServer.sh status
/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: followerㅁ Kafka 구동
## 서비스 시작 및 확인
[kafka@ka01 kafka]$ sudo bin/kafka-server-start.sh config/server.properties &
[1] 31000
[kafka@ka01 kafka]$ [2022-09-22 16:49:34,226] INFO Registered kafka:type=kafka.Log4jCont
roller MBean (kafka.utils.Log4jControllerRegistration$)
[2022-09-22 16:49:35,022] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=
true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
...
[2022-09-22 16:49:37,786] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)
[kafka@ka01 kafka]$
## 서비스 중지
[kafka@ka02 kafka]$ sudo bin/kafka-server-stop.sh
[2022-09-22 16:55:34,848] INFO Terminating process due to signal SIGTERM
(org.apache.kafka.common.utils.LoggingSignalHandler)
...
[2022-09-22 16:55:35,296] INFO [KafkaServer id=2] shut down completed
(kafka.server.KafkaServer)
[1]+ Exit 143 sudo bin/kafka-server-start.sh config/server.properties
[kafka@ka02 kafka]$
■ Kafka 메시지 전송 확인
[root@ka01 ~]# su - kafka
마지막 로그인: 목 9월 22 10:07:45 KST 2022 일시 pts/0
## 토픽 생성
[kafka@ka01 ~]$ /opt/kafka/bin/kafka-topics.sh --bootstrap-server ka01:9092 --replication-factor 1 --partitions 1 --create --topic test01
Created topic test01.
## Product 메시지 생성
[kafka@ka01 ~]$ /opt/kafka/bin/kafka-console-producer.sh --broker-list ka01:9092,ka02:9092,ka03:9092 --topic test01
>hello kafka
>nice to meet you
>^C
[kafka@ka01 ~]$ /opt/kafka/bin/kafka-console-producer.sh --broker-list ka01:9092,ka02:9092,ka03:9092 --topic test01
>i'm back again
>^C
## Consumer 메시지 획득
[kafka@ka01 ~]$ /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server ka01:9092 --topic test01 --from-beginning
hello kafka
nice to meet you
i'm back again
※ Resources
•https://jisootech.notion.site/jisootech/Kafka_3-2227884d67374f528357aca38298ef00
•https://docs.confluent.io/5.5.1/installation/index.html
•https://data-flair.training/blogs/
반응형