2025年docker-compose部署emqx集群 配置带mysql授权认证

docker-compose部署emqx集群 配置带mysql授权认证EMQX 是一款大规模可弹性伸缩的云原生分布式物联网 MQTT 消息服务器 作为全球最具扩展性的 MQTT 消息服务器 EMQX 提供了高效可靠海量物联网设备连接 能够高性能实时移动与处理消息和事件流数据 帮助您快速构建关键业务的物联网平台与应用 产品详细介绍 产品概览 EMQX 5 0 文档 https

大家好,我是讯享网,很高兴认识大家。

EMQX 是一款大规模可弹性伸缩的云原生分布式物联网 MQTT 消息服务器。

作为全球最具扩展性的 MQTT 消息服务器,EMQX 提供了高效可靠海量物联网设备连接,能够高性能实时移动与处理消息和事件流数据,帮助您快速构建关键业务的物联网平台与应用。


产品详细介绍:产品概览 | EMQX 5.0 文档
讯享网https://www.emqx.io/docs/zh/v5.0/


前言

EMQX 是目前全球市场广泛应用的百万级开源 MQTT 消息服务器,全球市场(西欧、北美、印度、中国)累积超 5000 家企业用户,产品环境下部署超 1 万节点,累计下载量超过 50 万,承载 MQTT 连接超 3000 万线

EMQ X 可以作为智能硬件、智能家居、物联网、车联网应用的百万级设备接入平台


基础配置就不介绍了,去官方看详细的配置,本文主要介绍下通过docker-compose的方式部署

  • docker已安装
  • docker-compose已安装

一、单机集群(伪集群)

创建 docker-compose.yaml

version: '3' services: emqx1: image: emqx/emqx:4.3.10 restart: always container_name: emqx1-borker1 deploy: resources: limits: cpus: "0.5" memory: 1G reservations: memory: 1G environment: - "EMQX_NAME=emqx" - "EMQX_HOST=node1.emqx.io" - "EMQX_CLUSTER__DISCOVERY=static" - "EMQX_CLUSTER__STATIC__SEEDS=, " - "EMQX_ALLOW_ANONYMOUS=false" - "EMQX_ACL_NOMATCH=deny" - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid" - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true" - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s" - "EMQX_AUTH__MYSQL__SERVER=192.168.1.1:3306" - "EMQX_AUTH__MYSQL__POOL=8" - "EMQX_AUTH__MYSQL__USERNAME=root" - "EMQX_AUTH__MYSQL__PASSWORD=root" - "EMQX_AUTH__MYSQL__DATABASE=emqx" - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=5s" - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256" - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password as password from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'" volumes: - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z ports: - 1883:1883 - 8081:8081 - 8083:8083 - 8084:8084 - 8883:8883 - 18083:18083 networks: emqx-bridge: aliases: - node1.emqx.io emqx2: image: emqx/emqx:4.3.10 environment: - "EMQX_NAME=emqx" - "EMQX_HOST=node2.emqx.io" - "EMQX_CLUSTER__DISCOVERY=static" - "EMQX_CLUSTER__STATIC__SEEDS=, " - "EMQX_ALLOW_ANONYMOUS=false" - "EMQX_ACL_NOMATCH=deny" - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid" - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true" - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s" - "EMQX_AUTH__MYSQL__SERVER=192.168.1.1:3306" - "EMQX_AUTH__MYSQL__POOL=8" - "EMQX_AUTH__MYSQL__USERNAME=root" - "EMQX_AUTH__MYSQL__PASSWORD=root" - "EMQX_AUTH__MYSQL__DATABASE=emqx" - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=15s" - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256" - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password_hash as password from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u'" - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'" volumes: - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z ports: - 1884:1883 networks: emqx-bridge: aliases: - node2.emqx.io networks: emqx-bridge: driver: bridge

讯享网

loaded_plugins文件:用于替换插件启用配置(启用emqx_auth_mysql插件)

讯享网{emqx_management, true}. {emqx_recon, true}. {emqx_retainer, true}. {emqx_dashboard, true}. {emqx_telemetry, true}. {emqx_rule_engine, true}. {emqx_bridge_mqtt, false}. {emqx_auth_mysql, true}. 

mysql授权认证表:

CREATE TABLE `iot_mqtt_acl` ( `id` int NOT NULL AUTO_INCREMENT COMMENT '主键', `username` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt用户名', `topic` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'Topic Filter', `access` int DEFAULT NULL COMMENT '访问权限 1: subscribe, 2: publish, 3: pubsub', `create_time` datetime DEFAULT NULL COMMENT '创建日期', `del_flag` int DEFAULT '0' COMMENT '是否删除 0未删除 1删除', PRIMARY KEY (`id`) USING BTREE, KEY `idx_username` (`username`) USING BTREE ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC COMMENT='mqtt的ACL规则表'; CREATE TABLE `iot_mqtt_user` ( `id` int NOT NULL AUTO_INCREMENT COMMENT '主键', `username` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt用户名', `password` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt加密的密码', `login_pwd` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt明文的密码', `is_superuser` tinyint(1) DEFAULT NULL COMMENT '是否超级用户(1-是,0-否)', `create_by` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL COMMENT '创建人', `create_time` datetime DEFAULT NULL COMMENT '创建日期', `update_by` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL COMMENT '更新人', `update_time` datetime DEFAULT NULL COMMENT '更新日期', `del_flag` int DEFAULT '0' COMMENT '是否删除 0未删除 1删除', PRIMARY KEY (`id`) USING BTREE, KEY `idx_username` (`username`) USING BTREE, KEY `idx_sn` (`sn`) USING BTREE ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC COMMENT='mqtt用户表';

 关于日志挂载

可以在yaml文件中配置 ./logs:/opt/emqx/log,说明我们把日志挂载出来了
但是这里有个问题:容器中使用的并不是root用户,如果我们用root用户启动容器,则docker在宿主机上创建的logs目录的属主是 root,那么容器中的服务会没有权限写,有如下两个解决方案:

  • 给logs目录777 权限(不推荐)
  • 修改logs目录属主(推荐)
    一般容器中的用户会映射到宿主机中 用户ID是 1001这个用户。为了保险我们可以自己实测:
    1)创建logs目录给777 权限
    2)启动服务,查看容器在logs目录下创建的文件属主(比如看到是用户A)
    3)停止容器,删除logs目录
    4)重新创建logs目录,将主、组都修改为用户A

启动: docker-compose up -d

注意:最开始没有在docker-compose文件中没有配置限制内容大小等,后面加的;如果使用docker-compose up会有问题(docker-compose 默认的有deploy配置),可以使用这个命令启动:

docker-compose不支持deploy节点配置启动命令:
docker-compose --compatibility up -d

查看启动的容器:

  • web访问
  • URL: http://IP:18083

emqx如果不需要授权的可以干掉授权部分的配置(参考:使用 MySQL 的密码认证 | EMQX 5.0 文档)

二、多态机子集群部署

1. 环境说明(IP是阿里云内网IP,这里只是展示,请自行切换)

节点 IP
emqx01 192.168.1.1
emqx02 192.168.1.2

emqx集群部署

2.1 emqx01

网络需要使用host模式

  • docker-compose.yml 文件
讯享网version: '3' services: emqx1: image: emqx/emqx:4.3.10 restart: always container_name: emqx1-borker deploy: resources: limits: cpus: "0.5" memory: 1G reservations: memory: 1G environment: - "EMQX_NAME=emqx" - "EMQX_HOST=192.168.1.1" - "EMQX_NODE_NMAE=emqx@192.168.1.1" - "EMQX_CLUSTER__DISCOVERY=static" - "EMQX_CLUSTER__STATIC__SEEDS=emqx@192.168.1.1,emqx@192.168.1.2" - "EMQX_CLUSTER_AUTOHEAL=on" - "EMQX_ALLOW_ANONYMOUS=false" - "EMQX_ACL_NOMATCH=deny" - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid" - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true" - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s" - "EMQX_AUTH__MYSQL__SERVER=域名/IP:3306" - "EMQX_AUTH__MYSQL__POOL=8" - "EMQX_AUTH__MYSQL__USERNAME=root" - "EMQX_AUTH__MYSQL__PASSWORD=xxxxx" - "EMQX_AUTH__MYSQL__DATABASE=emqx" - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=15s" - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256" - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password as password from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'" network_mode: host volumes: - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z healthcheck: test: ["CMD", "/opt/emqx/bin/emqx_ctl", "status"] interval: 5s timeout: 25s retries: 5 
  • 启动
[root@XXXXXX emqx01]# docker-compose --compatibility up -d 
  • 查看结果
讯享网[root@xxxx emqx01]# docker-compose ps Name Command State Ports ---------------------------------------------------------------------- emqx1_broker /usr/bin/docker-entrypoint ... Up (healthy)

2.2 emqx02

网络需要使用host模式

  • docker-compose.yml 文件
version: '3' services: emqx1: image: emqx/emqx:4.3.10 restart: always container_name: emqx2-borker deploy: resources: limits: cpus: "0.5" memory: 1G reservations: memory: 1G environment: - "EMQX_NAME=emqx" - "EMQX_HOST=192.168.1.1" - "EMQX_NODE_NMAE=emqx@192.168.1.2" - "EMQX_CLUSTER__DISCOVERY=static" - "EMQX_CLUSTER__STATIC__SEEDS=emqx@192.168.1.1,emqx@192.168.1.2" - "EMQX_CLUSTER_AUTOHEAL=on" - "EMQX_ALLOW_ANONYMOUS=false" - "EMQX_ACL_NOMATCH=deny" - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid" - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true" - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s" - "EMQX_AUTH__MYSQL__SERVER=域名/IP:3306" - "EMQX_AUTH__MYSQL__POOL=8" - "EMQX_AUTH__MYSQL__USERNAME=root" - "EMQX_AUTH__MYSQL__PASSWORD=xxxxx" - "EMQX_AUTH__MYSQL__DATABASE=emqx" - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=15s" - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256" - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password as password from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u' limit 1" - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'" network_mode: host volumes: - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z healthcheck: test: ["CMD", "/opt/emqx/bin/emqx_ctl", "status"] interval: 5s timeout: 25s retries: 5 
  • 启动
讯享网[root@XXXXXX emqx02]# docker-compose --compatibility up -d 
  • 查看结果
[root@xxxx emqx02]# docker-compose ps Name Command State Ports ---------------------------------------------------------------------- emqx2_broker /usr/bin/docker-entrypoint ... Up (healthy)

WEB查看结果:

IP1:18083

IP2:18083


总结

提示:集群和集群后同一个订阅消息花了几天时间,最后才发现是阿里云的内网端口的问题,明明是同一个地区,同一个专有网络,结果发现内网端口不通;最后才知道因为两台服务器安全组不一样。

进入容器查看集群状态: docker exec -it emqx1-borker sh -c "emqx_ctl cluster status"

同一个集群下的客户端,订阅相同主题缺无法收到消息:

4.0+版本,检查5369端口开放

集群方式介绍:

讯享网设置集群方式(manual) 手动集群方式是 EMQ X 的默认集群方式。 请在配置文件emqx/etc/emqx.conf中找到以下行并做相应修改: cluster.discovery = manual #-manual:手动加入命令 #-static:静态节点列表 #-mcast:IP组播 #-dns:DNS自动集群 #-etcd:etcd自动集群 #-k8s:Kubernetes

最开始集群未成功:还是端口的问题,导致我进容器手动添加:不推荐使用

$ ./bin/emqx_ctl cluster join  Join the cluster successfully. Cluster status: [{running_nodes,['','']}]

好了:以上就是今天要讲的内容,记录下自己踩过的坑

EMQX产品优势:

  • 开放源码:基于 Apache 2.0 许可证完全开源,自 2013 年起 200+ 开源版本迭代。
  • MQTT 5.0:100% 支持 MQTT 5.0 和 3.x 协议标准,更好的伸缩性、安全性和可靠性。
  • 海量连接:单节点支持 500 万 MQTT 设备连接,集群可扩展至 1 亿并发 MQTT 连接。
  • 高性能:单节点支持每秒实时接收、移动、处理与分发数百万条的 MQTT 消息。
  • 低时延:基于 Erlang/OTP 软实时的运行时系统设计,消息分发与投递时延低于 1 毫秒。
  • 高可用:采用 Masterless 的大规模分布式集群架构,实现系统高可用和水平扩展
小讯
上一篇 2025-03-10 22:20
下一篇 2025-02-16 17:10

相关推荐

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容,请联系我们,一经查实,本站将立刻删除。
如需转载请保留出处:https://51itzy.com/kjqy/35327.html