EMQX10万设备连接测试

EMQX 版本

5.7.2

EMQX 安装部署方式

emqx-5.7.2-el7-amd64.tar.安装包部署

EMQX 集群情况

单节点跟集群都尝试过
单节点配置如下:
node {
name = “emqx@172.29.247.179
cookie = “emqxsecretcookie”
data_dir = “data”
}
listeners.tcp.default {
bind = “0.0.0.0:1884”
max_connections = 1024000
}
zone {
external {
max_mqueue_len = 0
}
internal {
max_mqueue_len = 0
}
}
log {

}
dashboard {
listeners.http {
bind = 18083
}
}

集群配置文件如下:
node {
name = “emqx@172.29.247.172
cookie = “emqxsecretcookie”
data_dir = “data”
}
listeners.tcp.default {
bind = “0.0.0.0:1884”
max_connections = 1024000

proxy_protocol = true

}
zone {
external {
max_mqueue_len = 0
}
internal {
max_mqueue_len = 0
}
}
cluster {
discovery_strategy = static
static {
seeds = [“emqx@172.29.247.161”, “emqx@172.29.247.172”, “emqx@172.29.247.179”]
}
}
log {
}
dashboard {
listeners.http {
bind = 18083
}
}

服务器(运行 EMQX 的机器)硬件配置

8核 32GB,CPU 型号:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian

服务器操作系统和平台

centos7.5

测试场景

使用jemter工具,测试10万模拟设备连接到emqx,开始使用集群,使用nginx代理,发现最大连接只能连接64,512,
后面使用单机emqx,单机方案没有使用nginx代理,发现连接还是64,512,一直不能达到10万

具体问题

使用jemter工具,测试10万模拟设备连接到emqx,开始使用集群,使用nginx代理,发现最大连接只能连接64,512,
emqx日志如下:
2025-01-21T15:35:48.450928+08:00 [error] supervisor: {esockd_connection_sup,<0.111738.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1728}, offender: [{pid,<0.111738.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T15:45:06.750887+08:00 [error] supervisor: {esockd_connection_sup,<0.152762.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1085}, offender: [{pid,<0.152762.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T15:55:14.431611+08:00 [error] supervisor: {esockd_connection_sup,<0.232496.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 2083}, offender: [{pid,<0.232496.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:06:18.186762+08:00 [error] supervisor: {esockd_connection_sup,<0.325636.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1062}, offender: [{pid,<0.325636.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:07:07.658748+08:00 [error] supervisor: {esockd_connection_sup,<0.355126.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1192}, offender: [{pid,<0.355126.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:19:42.175628+08:00 [error] supervisor: {esockd_connection_sup,<0.383285.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1005}, offender: [{pid,<0.383285.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:19:43.802252+08:00 [error] supervisor: {esockd_connection_sup,<0.406123.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 2459}, offender: [{pid,<0.406123.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:31:26.241529+08:00 [error] supervisor: {esockd_connection_sup,<0.414067.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1105}, offender: [{pid,<0.414067.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:38:04.045374+08:00 [error] supervisor: {esockd_connection_sup,<0.617735.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1033}, offender: [{pid,<0.617735.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:47:09.025357+08:00 [error] supervisor: {esockd_connection_sup,<0.646658.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1098}, offender: [{pid,<0.646658.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:51:27.914670+08:00 [error] supervisor: {esockd_connection_sup,<0.719540.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1335}, offender: [{pid,<0.719540.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},enable_authn => true,zone => default}]}}]
2025-01-21T16:59:38.326997+08:00 [warning] msg: long_schedule, info: [{timeout,301},{in,{emqx_pmon,find,2}},{out,{emqx_cm,mark_channel_disconnected,1}}], procinfo: [{pid,<0.2829.0>},{memory,5758776},{total_heap_size,544598},{heap_size,514838},{stack_size,28},{min_heap_size,233},{proc_lib_initial_call,{emqx_cm,init,[‘Argument__1’]}},{initial_call,{proc_lib,init_p,5}},{current_stacktrace,[{emqx_utils,drain_down,2,[{file,“emqx_utils.erl”},{line,246}]},{emqx_cm,handle_info,2,[{file,“emqx_cm.erl”},{line,749}]},{gen_server,try_handle_info,3,[{file,“gen_server.erl”},{line,1095}]},{gen_server,handle_msg,6,[{file,“gen_server.erl”},{line,1183}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,241}]}]},{registered_name,emqx_cm},{status,running},{message_queue_len,424},{group_leader,<0.2686.0>},{priority,normal},{trap_exit,false},{reductions,761668},{last_calls,false},{catchlevel,2},{trace,0},{suspending,},{sequential_trace_token,},{error_handler,error_handler}]
2025-01-22T14:17:48.800211+08:00 [error] message=channel_closed driver=tcp socket=“#Port<0.24184>” action=stopping
2025-01-22T14:18:06.722376+08:00 [error] message=channel_closed driver=tcp socket=“#Port<0.24188>” action=stopping
2025-01-22T14:18:06.723182+08:00 [error] message=channel_closed driver=tcp socket=“#Port<0.2679>” action=stopping
2025-01-22T14:18:06.722846+08:00 [error] message=channel_closed driver=tcp socket=“#Port<0.25>” action=stopping
2025-01-22T14:19:39.279855+08:00 [warning] msg: cluster_config_fetch_failures, peer_nodes: [‘emqx@172.29.247.161’,‘emqx@172.29.247.172’], self_node: ‘emqx@172.29.247.179’, booting_nodes: , failed_nodes: [‘emqx@172.29.247.161’,‘emqx@172.29.247.172’]
2025-01-22T14:19:39.284891+08:00 [warning] msg: config_key_not_recognized, unknown_config_keys: zone
2025-01-22T16:14:25.313383+08:00 [error] supervisor: {esockd_connection_sup,<0.4575.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1055}, offender: [{pid,<0.4575.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:15:16.905297+08:00 [error] supervisor: {esockd_connection_sup,<0.211281.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 2539}, offender: [{pid,<0.211281.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:15:20.806786+08:00 [error] supervisor: {esockd_connection_sup,<0.211367.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1839}, offender: [{pid,<0.211367.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:16:04.841993+08:00 [error] supervisor: {esockd_connection_sup,<0.211374.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1024}, offender: [{pid,<0.211374.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:16:10.409328+08:00 [error] supervisor: {esockd_connection_sup,<0.211441.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 2273}, offender: [{pid,<0.211441.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:16:12.070989+08:00 [error] supervisor: {esockd_connection_sup,<0.211448.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1124}, offender: [{pid,<0.211448.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:16:57.353348+08:00 [error] supervisor: {esockd_connection_sup,<0.211455.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1040}, offender: [{pid,<0.211455.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:16:59.753270+08:00 [error] supervisor: {esockd_connection_sup,<0.211529.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 2428}, offender: [{pid,<0.211529.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:17:57.441697+08:00 [error] supervisor: {esockd_connection_sup,<0.211536.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1026}, offender: [{pid,<0.211536.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:18:46.434272+08:00 [error] supervisor: {esockd_connection_sup,<0.211627.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1309}, offender: [{pid,<0.211627.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:18:49.188834+08:00 [error] supervisor: {esockd_connection_sup,<0.211701.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1321}, offender: [{pid,<0.211701.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:18:52.297820+08:00 [error] supervisor: {esockd_connection_sup,<0.211708.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1847}, offender: [{pid,<0.211708.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:19:39.003335+08:00 [error] supervisor: {esockd_connection_sup,<0.211715.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1323}, offender: [{pid,<0.211715.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:20:29.115750+08:00 [error] supervisor: {esockd_connection_sup,<0.211790.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1127}, offender: [{pid,<0.211790.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:22:59.738835+08:00 [error] supervisor: {esockd_connection_sup,<0.211964.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1258}, offender: [{pid,<0.211964.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:23:54.803112+08:00 [error] supervisor: {esockd_connection_sup,<0.212251.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 2167}, offender: [{pid,<0.212251.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
2025-01-22T16:24:50.443227+08:00 [error] supervisor: {esockd_connection_sup,<0.212334.0>}, errorContext: connection_shutdown, reason: #{max => 1000,reason => mailbox_overflow,value => 1350}, offender: [{pid,<0.212334.0>},{name,connection},{mfargs,{emqx_connection,start_link,[#{listener => {tcp,default},limiter => #{connection => #{initial => 0,rate => infinity,burst => 0}},zone => default,enable_authn => true}]}}]
问一下,emqx有解决方案,使用jemter工具,测试10万模拟设备连接到emqx,最好有集群方案;或者告诉我,我该怎么修改


已经在这里回复您了,您描述中有 2 个问题,建议解决的方向是:

  1. 无法达到 10W 的连接数。这个可以直接搜索 emqx 系统调优。 找到 linux 的调整参数优化。
  2. mailbox_overflow 的问题是您设计的 topic 订阅了太多的消息,消费不了。可以考虑使用共享订阅来分担。

同时你这个配置是无效的。
正确的设置方法是:

mqtt.max_mqueue_len=infinity

不过非常不建议在生产环境设置为 infinity。