WEB集群部署
1. WEB集群部署
服务器资源分配
| 服务器角色 | 服务器 A | 服务器 B |
|---|---|---|
| Nginx 负载均衡 | ❌（单点部署于 B） | ✅（主负载均衡） |
| Spring Boot 应用 | ✅（Fastbee，含 Ignite 节点） | ✅（Fastbee，含 Ignite 节点） |
| 数据库 | ✅（MySQL + Tdengine） | 主从模式待实现.... |
| 缓存 | ✅（Redis 单点，带持久化） | 主从架构待实现... |
1. 环境准备（两台服务器均执行）
安装 Docker 和 Docker Compose
bash# 安装Docker curl -fsSL https://get.docker.com | sh systemctl start docker && systemctl enable docker # 安装Docker Compose curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose
2. 服务器 A 部署：数据库、Redis、Fastbee服务
假设两台服务器A和B，公网IP：123.0.0.1 和 124.0.0.2 内网: 10.1.1.10 和 20.2.2.20
打包后端jar包
fastbee-admin.jar 打包前端
dist包配置docker-compose文件
按照文档提示进行修改
ymlversion: '3.8' services: redis: image: redis:7.0.0 container_name: redis ports: - 6379:6379 privileged: true network_mode: host volumes: - /var/data/redis:/usr/local/etc/redis - /var/data/redis/data:/data command: [ '--requirepass', 'admin123', '--appendonly', 'yes' ] #密码是admin123,自行修改 mysql: image: mysql:5.7 container_name: mysql environment: MYSQL_ROOT_PASSWORD: admin123 #密码是admin123,自行修改 MYSQL_DATABASE: fastbee #数据库名，自行更改 ports: - 3306:3306 privileged: true network_mode: host volumes: - /var/data/mysql/data:/var/lib/mysql - /var/data/mysql/initdb:/docker-entrypoint-initdb.d command: [ 'mysqld', '--character-set-server=utf8', '--collation-server=utf8_unicode_ci', '--default-time-zone=+8:00', '--lower-case-table-names=1', '--bind-address=0.0.0.0' ] # 假设两台服务器A和B，公网IP：123.0.0.1 和 124.0.0.2 内网: 10.1.1.10 和 20.2.2.20 java: image: openjdk:8-jre container_name: java ports: - 8080:8080 - 47500-47509:47500-47509 - 47101:47101 - 11211:11211 - 10800:10800 - 48500:48500 - 48880:48880 environment: WORKER_HOST: 123.0.0.1 # 服务器A公网IP 或内网IP IGNITE_LOCAL_IP: 10.1.1.10 # 服务器A内网IP IGNITE_CLUSTER_NODES: 10.1.1.10,20.2.2.20 # 服务器A和B内网IP network_mode: host volumes: - /var/data/java/fastbee-admin.jar:/server.jar - /var/data/java/license:/license - /var/data/java/libtaos.so:/usr/lib/libtaos.so - /var/data/java/uploadPath:/uploadPath - /var/data/java/logs:/logs - /etc/localtime:/etc/localtime depends_on: - redis - mysql - tdengine entrypoint: - sh - -c - | exec java \ -Xmx4g -Xms4g \ #4核8G配置,根据情况调优 -DIGNITE_QUIET=false \ -DIGNITE_NO_DISCO_ORDER=true \ -Djava.net.preferIPv4Stack=true \ -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \ -XX:+UseG1GC -XX:G1HeapRegionSize=32m \ -XX:ParallelGCThreads=4 -XX:ConcGCThreads=2 \ -Dfile.encoding=UTF-8 \ -Dignite.heap.offheap.size=4g \ #4核8G配置,根据情况调优 -jar /server.jar tdengine: image: tdengine/tdengine:3.3.2.0 hostname: tdengine container_name: tdengine ports: - 6030:6030 - 6041:6041 - 6043-6049:6043-6049 - 6043-6049:6043-6049/udp network_mode: host volumes: - /var/data/tdengine/log:/var/log/taos - /var/data/tdengine/data:/var/lib/taos - /var/data/tdengine/conf:/etc/taos - /etc/localtime:/etc/localtime
3. 服务器 B 部署：Nginx 负载均衡、Spring Boot 应用
nginx.conf负载均衡配置
apl
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
use epoll;
multi_accept on;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
keepalive_requests 1000;
client_max_body_size 100m;
gzip on;
gzip_min_length 1k;
gzip_buffers 16 64k;
gzip_http_version 1.1;
gzip_comp_level 5;
gzip_types text/plain application/javascript application/x-javascript text/javascript text/css application/xml;
gzip_vary on;
gzip_proxied expired no-cache no-store private auth;
gzip_disable "MSIE [1-6]\.";
# 服务器A和B的内网IP 自行更改
upstream backend {
server 10.1.1.10:8080 max_fails=1 fail_timeout=10s; # 服务A内网IP,自行修改
server 20.2.2.20:8080 max_fails=1 fail_timeout=10s; # 服务B内网IP,自行修改
keepalive 60;
least_conn;
}
server {
listen 80;
server_name localhost;
charset utf-8;
# SSL 默认访问端口号为443
listen 443 ssl;
server_name localhost;
charset utf-8;
# 证书文件的路径
ssl_certificate /usr/share/nginx/ssl/fastbee.crt;
# 私钥文件的路径
ssl_certificate_key /usr/share/nginx/ssl/fastbee.key;
ssl_session_timeout 10m;
# 请按照以下协议配置
ssl_protocols TLSv1.2 TLSv1.3;
# 请按照以下套件配置，配置加密套件，写法遵循openssl 标准
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
ssl_prefer_server_ciphers on;
#................其他配置
location /prod-api/ {
proxy_pass http://backend/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE-HOST $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Connection "";
proxy_redirect off;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
send_timeout 300s;
}
}
}docker-compose配置修改
yml
version: '3.8'
services:
nginx:
image: nginx:stable
container_name: nginx
ports:
- "80:80"
- "443:443"
volumes:
- /var/data/nginx/vue:/usr/share/nginx/html
- /var/data/nginx/view:/usr/share/nginx/view
- /var/data/nginx/h5:/usr/share/nginx/h5
# - /var/data/nginx/ssl:/usr/share/nginx/ssl #SSL证书有则放开
- /var/data/nginx/logs:/var/log/nginx
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
network_mode: host
# 假设两台服务器A和B，公网IP：123.0.0.1 和 124.0.0.2 内网: 10.1.1.10 和 20.2.2.20
java:
image: openjdk:8-jre
container_name: java
ports:
- 8080:8080
- 47500-47509:47500-47509
- 47101:47101
- 11211:11211
- 10800:10800
- 48500:48500
- 48880:48880
environment:
WORKER_HOST: 124.0.0.2 # 服务器A公网IP 或内网IP
IGNITE_LOCAL_IP: 20.2.2.20 # 服务器A内网IP
IGNITE_CLUSTER_NODES: 10.1.1.10,20.2.2.20 # 服务器A和B内网IP
volumes:
- /var/data/java/fastbee-admin.jar:/server.jar
- /var/data/java/license:/license
- /var/data/java/libtaos.so:/usr/lib/libtaos.so
- /var/data/java/uploadPath:/uploadPath
- /var/data/java/logs:/logs
- /etc/localtime:/etc/localtime
network_mode: host
entrypoint:
- sh
- -c
- |
exec java \
-Xmx4g -Xms4g \
-DIGNITE_QUIET=false \
-DIGNITE_NO_DISCO_ORDER=true \
-Djava.net.preferIPv4Stack=true \
-XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \
-XX:+UseG1GC -XX:G1HeapRegionSize=32m \
-XX:ParallelGCThreads=4 -XX:ConcGCThreads=2 \
-Dfile.encoding=UTF-8 \
-Dignite.heap.offheap.size=4g \
-jar /server.jar4. 服务目录
按照上面修改好后
- 将项目中 docker目录下面，ServerA中 data复制至 服务器A
/var目录下 - 将项目中 docker目录下面，ServerB中 data复制至 服务器B
/var目录下
执行目录授权
sh
cd /var/data
setenforce 0
chmod 777 -R /var/data5.部署
在服务器A /var/data 执行
sh
docker-compose up -d
# 查看部署情况
#[root@VM-xxx data]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
429020770a4d openjdk:8-jre "sh -c 'exec java \\\n…" 16 minutes ago Up 16 minutes java
eebb0c7c40e9 tdengine/tdengine:3.3.2.0 "/tini -- /usr/bin/e…" 16 minutes ago Up 16 minutes tdengine
501697f9bfa7 redis:7.0.0 "docker-entrypoint.s…" 16 minutes ago Up 16 minutes redis
0b31966146f5 mysql:5.7 "docker-entrypoint.s…" 16 minutes ago Up 16 minutes mysql在服务器B /var/data 执行
sh
docker-compose up -d
# 查看部署情况
#[root@VM-xxx data]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9e7622669ec5 openjdk:8-jre "sh -c 'exec java \\\n…" About a minute ago Up About a minute java
4ed9ca408461 nginx:stable "/docker-entrypoint.…" About a minute ago Up About a minute nginx6.kafka负载均衡
1.分区与副本机制
数据分片：每个topic被划分到多个分区，部分在不同Broker上，生产者发送消息，通过分区策略将消息均匀分配到各分区，实现写入负载均衡
副本备份: 每个分区有多个副本，Leader处理读写请求， Follower同步数据，Leader故障，Follower替换
2. 消费者组
同一消费者组内的消费者通过分配策略（Round Robin，Range），分摊分区消费任务，例如通过轮训分配，确保同一消费者组中分区数据平衡
3.实施(重点)
kafka的分区无法在代码实现，需要借助外部工具，例如之前提到的 kafka-console-ui
下面举例实现数据上报的消费负载均衡：
目前数据上报主题 fmq_property_post 消费实例是 10个，假设现在集群部署两个web服务节点，该主题的消费实例总数是 20，那么要达到数据上报消费负载均衡需要满足 分区数 >= 所有节点主题消费实例总数
更改分区数
验证kafka负载均衡
生产者发送数据，查看 消费者组 - 消费情况 -每个分区的消费是否有并相对均衡
7.验证Ignite集群
方式1：查看java容器 sys-info.log
Cluster [hosts=2, CPUs=8, servers=2, clients=0, topVer=4, minorTopVer=1] hosts =2 表示集群节点为2
ruby
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=592e782c, name=fastbee-server, uptime=03:37:01.004]
^-- Cluster [hosts=2, CPUs=8, servers=2, clients=0, topVer=4, minorTopVer=1]
^-- Network [addrs=[10.0.24.6], localHost=10.0.24.6, discoPort=47501, commPort=47101]
^-- CPU [CPUs=4, curLoad=0.83%, avgLoad=1.04%, GC=0%]
^-- Heap [used=612MB, free=85.05%, comm=4096MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=0, qSize=0]
^-- System thread pool [active=0, idle=4, qSize=0]
^-- Striped thread pool [active=0, idle=8, qSize=0]- 方式2：可视化界面
待开发....
8.后端负载均衡验证
方式1：在 Spring Boot 中添接口打印请求来源 IP：
定位ToolController，后端全路径为
iot/tool/test
java
@GetMapping("/test")
public String test(HttpServletRequest request) {
return "接收到请求，来源 IP: " + request.getRemoteAddr();
}- 方式2：停止某节点 JAVA容器服务
sh
docker stop java前端访问无出现 正常，即验证成功