emqx性能测试

EMQX
1

emqx测试百万连接，两台服务器构建broker集群，用一台压力机启用一张网卡，并开启6万个连接数，但是实际到3，4万就崩掉了。参数调优消息如下
image
image
不知道是什么原因

2

image
image501×576 5.78 KB

中途就出现这个问题

3
  • 尝试降低一些 emqtt_bench 的连接速率
  • EMQX 版本及日志请提供下
  • EMQX 节点机器配置？请参考 配置估算
4

两台broker是16G内存，12核的

5

现在连接速率是1s创建100个连接

6

版本5.0.8

7

5m47s connect_succ total=34130 rate=98.00/sec
5m48s connect_succ total=34229 rate=99.00/sec
5m49s connect_succ total=34327 rate=97.61/sec
client(4269): connect error - econnrefused
client(4269): EXIT for {shutdown,econnrefused}
client(34269): connect error - econnrefused
client(34269): EXIT for {shutdown,econnrefused}
client(44269): connect error - econnrefused
client(44269): EXIT for {shutdown,econnrefused}
client(24269): connect error - econnrefused
client(24269): EXIT for {shutdown,econnrefused}
client(14269): connect error - econnrefused
client(14269): EXIT for {shutdown,econnrefused}
client(19186): EXIT for {shutdown,tcp_closed}
client(54268): connect error - econnrefused
client(54268): EXIT for {shutdown,econnrefused}
client(9186): EXIT for {shutdown,tcp_closed}
client(59186): EXIT for {shutdown,tcp_closed}
client(29186): EXIT for {shutdown,tcp_closed}
client(39186): EXIT for {shutdown,tcp_closed}
client(49186): EXIT for {shutdown,tcp_closed}
client(4268): connect error - econnrefused
client(4268): EXIT for {shutdown,econnrefused}
client(34268): connect error - econnrefused
client(34268): EXIT for {shutdown,econnrefused}
client(44268): connect error - econnrefused
client(44268): EXIT for {shutdown,econnrefused}
client(24268): connect error - econnrefused
client(24268): EXIT for {shutdown,econnrefused}
client(14268): connect error - econnrefused
client(14268): EXIT for {shutdown,econnrefused}
client(54267): connect error - econnrefused
client(54267): EXIT for {shutdown,econnrefused}
client(59185): EXIT for {shutdown,tcp_closed}
client(29185): EXIT for {shutdown,tcp_closed}
client(39185): EXIT for {shutdown,tcp_closed}
client(9185): EXIT for {shutdown,tcp_closed}
client(19185): EXIT for {shutdown,tcp_closed}
client(49185): EXIT for {shutdown,tcp_closed}
client(4267): connect error - econnrefused
client(4267): EXIT for {shutdown,econnrefused}
client(34267): connect error - econnrefused
client(34267): EXIT for {shutdown,econnrefused}
这是终端打印的信息

8

请问，哪些参数是会影响连接数？

9

您好。请提供一下emqx log。谢谢。

10

压测使用的 emqtt_bench 命令也请提供下
有可能是主机内核参数限制/bench机器参数限制。
我用纯 docker 环境测试双节点集群可以连到 250000 连接

docker run -it emqx/emqtt-bench conn -c 25000 -i 5 -h 172.17.0.6

image
image1419×656 62.2 KB

（emqx 使用容器，emqtt_bench 也为容器，主机配置 8c16t, 40GB 内存）

$ dclsa                                
CONTAINER ID   IMAGE              COMMAND                   CREATED             STATUS                      PORTS                                                                                   NAMES
516ea003ca18   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   40 minutes ago      Up 40 minutes                                                                                                       dreamy_turing
abd0a987b4da   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   40 minutes ago      Up 40 minutes                                                                                                       clever_haslett
77e44bff1516   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   41 minutes ago      Up 41 minutes                                                                                                       jolly_driscoll
1b0a99b1bea5   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   41 minutes ago      Up 41 minutes                                                                                                       relaxed_napier
bf1c23c7daee   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   43 minutes ago      Up 43 minutes                                                                                                       agitated_benz
57de4c85a7bc   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   43 minutes ago      Up 43 minutes                                                                                                       kind_wright
d2d4598fa02b   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   44 minutes ago      Up 44 minutes                                                                                                       strange_ritchie
2b8d52d06a33   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   44 minutes ago      Up 44 minutes                                                                                                       elegant_herschel
1316411c23a8   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   46 minutes ago      Up 46 minutes                                                                                                       cool_mayer
c68a419acda5   emqx/emqtt-bench   "/emqtt_bench/emqtt_…"   46 minutes ago      Up 46 minutes                                                                                                       elegant_feynman
fb7b5f2bc05a   emqx/emqx:5.0.8    "/usr/bin/docker-ent…"   About an hour ago   Up About an hour            1883/tcp, 4370/tcp, 5369/tcp, 8081/tcp, 8083-8084/tcp, 8883/tcp, 11883/tcp, 18083/tcp   emqx-5.0.8-1
52d872dfefa9   emqx/emqx:5.0.8    "/usr/bin/docker-ent…"   About an hour ago   Up About an hour            1883/tcp, 4370/tcp, 5369/tcp, 8081/tcp, 8083-8084/tcp, 8883/tcp, 11883/tcp, 18083/tcp   emqx-5.0.8-0

每个 emqtt_bench 只开启 25000 个连接是因为 emqtt_bench 容器内核参数限制连接数：

root@516ea003ca18:/emqtt_bench# sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768	60999
11

emqtt_bench conn -h 10.20.14.18 -c 60000 -i 10 --ifaddr 10.20.14.140

12

image

image
第一个是broker的，第二是压力机的

13
  • 建议你再调查下环境的问题，连接数我们是测试过的。
    一般有这种问题都是部署环境测试机/压力机的问题。
  • 尝试升级下最新的 v5.3.0
  • EMQX 的日志（如果换 v5.3.0 需要新版本上压力测试的日志）
14

请问，可以发一下推荐的具体的节点和压力机的参数配置信息吗，改完参数需要重启emqx服务吗，broker和压力机的参数可以设置成一样吗