测试mqtt性能的时候,mqtt发生了报错

2023-12-14T14:05:03.927597+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node1’, reason: {erpc,timeout}
2023-12-14T14:05:03.927786+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node3’, reason: {erpc,timeout}
2023-12-14T14:05:03.927864+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node2’, reason: {erpc,timeout}
2023-12-14T14:05:13.929055+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node1’, reason: {erpc,timeout}
2023-12-14T14:05:13.929273+08:00 [warning] trie-compaction flag is inconsistent, enabled: [‘emqx@node2’,‘emqx@node3’], disabled:x@node1’]
2023-12-14T14:05:13.929389+08:00 [warning] Trying to set ‘true’ for trie-compaction flag in all nodes.
2023-12-14T14:08:14.636175+08:00 [warning] [Alarm Handler] Alarm high_system_memory_usage is deactivated
2023-12-14T14:13:54.226028+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node3’, reason: {erpc,timeout}
2023-12-14T14:13:54.226213+08:00 [warning] trie-compaction flag is inconsistent, enabled: [‘emqx@node2’,‘emqx@node1’], disabled: @node3’]
2023-12-14T14:13:54.226306+08:00 [warning] Trying to set ‘true’ for trie-compaction flag in all nodes.
2023-12-14T14:20:05.756020+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node1’, reason: {erpc,timeout}
2023-12-14T14:20:05.756242+08:00 [warning] trie-compaction flag is inconsistent, enabled: [‘emqx@node2’,‘emqx@node3’], disabled: [‘emqx@node1’]
2023-12-14T14:20:05.756338+08:00 [warning] Trying to set ‘true’ for trie-compaction flag in all nodes.
2023-12-14T14:22:14.780088+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node1’, reason: {erpc,timeout}
2023-12-14T14:22:14.780264+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node3’, reason: {erpc,timeout}
2023-12-14T14:22:14.780332+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node2’, reason: {erpc,timeout}

以上是测试emqx 4
3个节点进行连接测试的时候,超过10w连接数的时候,很多连接断开了。请问emqx4单台节点最多可以连接多少个客户端呢? 并且以上这个报错应该如何解决?

最新的报错

le,“proc_lib.erl”},{line,236}]}]}]
2023-12-14T14:59:00.560640+08:00 [error] kill <0.21362.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_20772_1304101260”>>
2023-12-14T14:59:00.560662+08:00 [error] kill <0.26903.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_35833_98924953”>>
2023-12-14T14:59:00.560684+08:00 [error] kill <0.18130.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_1218_433753265”>>
2023-12-14T14:59:00.560684+08:00 [error] kill <0.11475.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_38535_2829302455”>>
2023-12-14T14:59:00.560568+08:00 [error] lock_owner_status:, [{status,suspended},{message_queue_len,2},{current_stacktrace,[{mnesia_tm,async_send_dirty,6,[{file,“mnesia_tm.erl”},{line,2012}]},{mnesia_tm,dirty,2,[{file,“mnesia_tm.erl”},{line,1076}]},{emqx_cm,register_channel,3,[{file,“emqx_cm.erl”},{line,150}]},{emqx_cm,‘-open_session/3-fun-0-’,5,[{file,“emqx_cm.erl”},{line,244}]},{emqx_cm_locker,trans,3,[{file,“emqx_cm_locker.erl”},{line,55}]},{emqx_channel,process_connect,2,[{file,“emqx_channel.erl”},{line,535}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,706}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,409}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,415}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,373}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,236}]}]}]
2023-12-14T14:59:00.560711+08:00 [error] kill <0.25751.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_68696_4021648298”>>
2023-12-14T14:59:00.560680+08:00 [error] lock_owner_status:, [{status,suspended},{message_queue_len,2},{current_stacktrace,[{mnesia_tm,async_send_dirty,6,[{file,“mnesia_tm.erl”},{line,2012}]},{mnesia_tm,dirty,2,[{file,“mnesia_tm.erl”},{line,1076}]},{emqx_cm,register_channel,3,[{file,“emqx_cm.erl”},{line,150}]},{emqx_cm,‘-open_session/3-fun-0-’,5,[{file,“emqx_cm.erl”},{line,244}]},{emqx_cm_locker,trans,3,[{file,“emqx_cm_locker.erl”},{line,55}]},{emqx_channel,process_connect,2,[{file,“emqx_channel.erl”},{line,535}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,706}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,409}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,415}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,373}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,236}]}]}]
2023-12-14T14:59:00.560759+08:00 [error] kill <0.21882.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_14188_2880274507”>>
2023-12-14T14:59:00.560781+08:00 [error] kill <0.24654.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_44694_1291980136”>>
2023-12-14T14:59:00.560800+08:00 [error] kill <0.23201.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_3197_895258260”>>
2023-12-14T14:59:00.560760+08:00 [error] kill <0.31032.77> as it has held the lock for too long, resource: <<“gf-test-store-0001_bench_pub_42231_3232008563”>>

使用的 emqx 版本号是企业版 4.4.21 吗?

表示 rpc 超时,可能是节点访问不通,也可能是 mnesia 过载。后面其他的错误大部分跟这个问题相关。

你压力测试的时候,连接速率是多少,每个连接建立完成后会马上订阅一个什么样的主题吗?

使用的是企业版 4.4.23

用的是官方的测试工具,
emqtt_bench pub -t t -h 192.168.0.1 -p 1883 -s 16 -q 0 -c 70000 -i1 -I 1000

,每秒连接1000个左右, 会直接在一个空白的主题上发消息

没有订阅吗?

先重启集群,确保集群状态正常,没有这种问题之后,再试一次压测:

2023-12-14T14:05:03.927597+08:00 [error] Failed to read trie-compaction config on node ‘emqx@node1’, reason: {erpc,timeout}