问题描述
我在windows server 2008的服务器上安装了VMWare 15,然后创建了1个Broker,两个压测客户端
使用emqtt-bench进行连接数压测,发现最多只能创建20万个连接,再多就会被强制Killed
我对每个客户端设置了10个网卡,都使用NAT网络
我用其中一个客户端,打开多个ssh做压测时,当连接数达到20万时,就会强制关闭压测程序,在不同的客户端上试验,结果也是一样,我按文档中的系统调优进行了服务端与客户端设置,请问这到底是哪方面的问题?
环境信息
- EMQ X 版本:emqx-centos7-4.2.10-x86_64 开源版
- 操作系统及版本:Windows server 2008 VMWare 15 CentOS7.8
- Broker:8核16G
- 客户端:4核8G
相应的配置文件内容
系统优化参考文档设置
详细日志
控制台输出
[root@centos emqtt-bench]# ./emqtt_bench conn -h 192.168.132.129 -c 50000 --ifaddr 192.168.132.140
connected: 1962client(49619): connect error - {{shutdown,tcp_closed},
{tcp_closed,#Port<0.387>}}
client(49925): connect error - {{shutdown,tcp_closed},
{tcp_closed,#Port<0.81>}}
connected: 34618client(15383): connect error - {{shutdown,tcp_closed},
{tcp_closed,#Port<0.34623>}}
connected: 49997Killed
erlang.log.1
=====
===== LOGGING STARTED Mon Apr 19 11:57:03 CST 2021
=====
Exec: /opt/emqx/erts-10.7/bin/erlexec -boot /opt/emqx/releases/4.2.10/emqx -mode embedded -boot_var ERTS_LIB_DIR /opt/emqx/erts-10.7/../lib -mnesia dir "/opt/emqx/data/mnesia/emqx@192.168.132.129" -config /opt/emqx/data/configs/app.2021.04.19.11.57.12.config -args_file /opt/emqx/data/configs/vm.2021.04.19.11.57.12.args -vm_args /opt/emqx/data/configs/vm.2021.04.19.11.57.12.args -start_epmd false -epmd_module ekka_epmd -proto_dist ekka -- console
Root: /opt/emqx
/opt/emqx
Erlang/OTP 22 [erts-10.7] [source] [64-bit] [smp:8:8] [ds:8:8:8] [async-threads:4] [hipe]
Starting emqx on node emqx@192.168.132.129
Start http:management listener on 8081 successfully.
Start http:dashboard listener on 18083 successfully.
Start mqtt:tcp listener on 127.0.0.1:11883 successfully.
Start mqtt:tcp listener on 0.0.0.0:1883 successfully.
Start mqtt:ws listener on 0.0.0.0:8083 successfully.
Start mqtt:ssl listener on 0.0.0.0:8883 successfully.
Start mqtt:wss listener on 0.0.0.0:8084 successfully.
EMQ X Broker 4.2.10 is running now!
Eshell V10.7 (abort with ^G)
(emqx@192.168.132.129)1>
===== ALIVE Mon Apr 19 12:12:37 CST 2021
===== ALIVE Mon Apr 19 12:27:37 CST 2021
===== ALIVE Mon Apr 19 12:42:37 CST 2021
===== ALIVE Mon Apr 19 12:57:37 CST 2021
===== ALIVE Mon Apr 19 13:12:37 CST 2021
===== ALIVE Mon Apr 19 13:27:37 CST 2021
===== Mon Apr 19 13:41:44 CST 2021
2021-04-19 13:41:44.808 [warning] [SYSMON] long_schedule warning: port = #Port<0.26>, info: [{timeout,435},
{port_op,dist_cmd}]
[{name,"tcp_inet"},
{links,[<0.1843.0>,<0.1305.0>]},
{id,208},
{connected,<0.1843.0>},
{input,1068003},
{output,51},
{os_pid,undefined}]
===== ALIVE Mon Apr 19 13:56:44 CST 2021
===== ALIVE Mon Apr 19 14:11:45 CST 2021
===== Mon Apr 19 14:21:44 CST 2021
2021-04-19 14:21:44.673 [warning] [SYSMON] busy_dist_port warning: suspid = <0.1556.0>, port = #Port<0.26>
[{initial_call,{proc_lib,init_p,5}},
{current_function,{erlang,bif_return_trap,2}},
{registered_name,emqx_pool_7},
{status,suspended},
{message_queue_len,6},
{group_leader,<0.1424.0>},
{priority,normal},
{trap_exit,false},
{reductions,192510},
{last_calls,false},
{catchlevel,3},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,700416},
{total_heap_size,87358},
{heap_size,10958},
{stack_size,37},
{min_heap_size,233}]
[{name,"tcp_inet"},
{links,[<0.1843.0>,<0.1305.0>]},
{id,208},
{connected,<0.1843.0>},
{input,34195357},
{output,51},
{os_pid,undefined}]
2021-04-19 14:23:04.185 [warning] [SYSMON] large_heap warning: pid = <0.1601.0>, info: [{old_heap_block_size,8912793},
{heap_block_size,1199557},
{mbuf_size,0},
{stack_size,32},
{old_heap_size,3650620},
{heap_size,347835}]
[{initial_call,{proc_lib,init_p,5}},
{current_function,{emqx_misc,drain_down,2}},
{registered_name,emqx_cm},
{status,running},
{message_queue_len,10260},
{group_leader,<0.1424.0>},
{priority,normal},
{trap_exit,false},
{reductions,19597911},
{last_calls,false},
{catchlevel,2},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,96709368},
{total_heap_size,10385191},
{heap_size,1199557},
{stack_size,27},
{min_heap_size,233}]
2021-04-19 14:25:49.193 [warning] [SYSMON] long_schedule warning: pid = <0.9295.31>, info: [{timeout,359},
{in,{gen_server,rec_nodes,7}},
{out,
{emqx_connection,recvloop,
2}}]
[{initial_call,{proc_lib,init_p,5}},
{current_function,{emqx_connection,recvloop,2}},
{registered_name,[]},
{status,waiting},
{message_queue_len,0},
{group_leader,<0.1397.0>},
{priority,normal},
{trap_exit,false},
{reductions,2208},
{last_calls,false},
{catchlevel,1},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,34556},
{total_heap_size,4184},
{heap_size,2586},
{stack_size,9},
{min_heap_size,233}]
2021-04-19 14:25:49.201 [warning] [SYSMON] long_schedule warning: pid = <0.3272.22>, info: [{timeout,360},
{in,undefined},
{out,
{emqx_connection,recvloop,
2}}]
[{initial_call,{proc_lib,init_p,5}},
{current_function,{emqx_connection,recvloop,2}},
{registered_name,[]},
{status,waiting},
{message_queue_len,0},
{group_leader,<0.1397.0>},
{priority,normal},
{trap_exit,false},
{reductions,2656},
{last_calls,false},
{catchlevel,1},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,9028},
{total_heap_size,993},
{heap_size,376},
{stack_size,9},
{min_heap_size,233}]
===== ALIVE Mon Apr 19 14:40:49 CST 2021
===== ALIVE Mon Apr 19 14:55:49 CST 2021
emqx.log
2021-04-19 14:21:44.673 [warning] [SYSMON] busy_dist_port warning: suspid = <0.1556.0>, port = #Port<0.26>
[{initial_call,{proc_lib,init_p,5}},
{current_function,{erlang,bif_return_trap,2}},
{registered_name,emqx_pool_7},
{status,suspended},
{message_queue_len,6},
{group_leader,<0.1424.0>},
{priority,normal},
{trap_exit,false},
{reductions,192508},
{last_calls,false},
{catchlevel,3},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,700416},
{total_heap_size,87358},
{heap_size,10958},
{stack_size,37},
{min_heap_size,233}]
[{name,"tcp_inet"},
{links,[<0.1843.0>,<0.1305.0>]},
{id,208},
{connected,<0.1843.0>},
{input,34195357},
{output,51},
{os_pid,undefined}]
2021-04-19 14:23:04.185 [warning] [SYSMON] large_heap warning: pid = <0.1601.0>, info: [{old_heap_block_size,8912793},
{heap_block_size,1199557},
{mbuf_size,0},
{stack_size,32},
{old_heap_size,3650620},
{heap_size,347835}]
[{initial_call,{proc_lib,init_p,5}},
{current_function,{emqx_misc,drain_down,2}},
{registered_name,emqx_cm},
{status,running},
{message_queue_len,2722},
{group_leader,<0.1424.0>},
{priority,normal},
{trap_exit,false},
{reductions,19393911},
{last_calls,false},
{catchlevel,2},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,95488164},
{total_heap_size,10220873},
{heap_size,1199557},
{stack_size,27},
{min_heap_size,233}]
2021-04-19 14:25:49.193 [warning] [SYSMON] long_schedule warning: pid = <0.9295.31>, info: [{timeout,359},
{in,{gen_server,rec_nodes,7}},
{out,
{emqx_connection,recvloop,
2}}]
[{initial_call,{proc_lib,init_p,5}},
{current_function,{emqx_connection,recvloop,2}},
{registered_name,[]},
{status,waiting},
{message_queue_len,0},
{group_leader,<0.1397.0>},
{priority,normal},
{trap_exit,false},
{reductions,2206},
{last_calls,false},
{catchlevel,1},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,34556},
{total_heap_size,4184},
{heap_size,2586},
{stack_size,9},
{min_heap_size,233}]
2021-04-19 14:25:49.201 [warning] [SYSMON] long_schedule warning: pid = <0.3272.22>, info: [{timeout,360},
{in,undefined},
{out,
{emqx_connection,recvloop,
2}}]
[{initial_call,{proc_lib,init_p,5}},
{current_function,{emqx_connection,recvloop,2}},
{registered_name,[]},
{status,waiting},
{message_queue_len,0},
{group_leader,<0.1397.0>},
{priority,normal},
{trap_exit,false},
{reductions,2654},
{last_calls,false},
{catchlevel,1},
{trace,0},
{suspending,[]},
{sequential_trace_token,[]},
{error_handler,error_handler},
{memory,9028},
{total_heap_size,993},
{heap_size,376},
{stack_size,9},
{min_heap_size,233}]