check_oom导致后端数据处理客户端重复断连

EMQX
,
1

做jmeter压测时，出现共享订阅服务器后端处理的MQTT客户端出现重复断连情况，日志追踪后出现如下DEBUG 主要是check_oom，请问这种情况如何避免？如何保证服务端处理的MQTT客户端能始终保持优先连接进行数据消费 避免在IOT设备连入过多时挤掉服务端数据处理的MQTT客户端

2024-04-28T02:38:15.074368+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:51772, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:15.176765+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:51772, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:15.271865+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:51772, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:15.376595+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:51772, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:19.327566+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:53855 msg: authenticator_result, authenticator: password_based:built_in_database, result: {ok,#{is_superuser => true}}
2024-04-28T02:38:19.328083+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:53855 msg: authentication_result, reason: chain_result, result: {stop,{ok,#{is_superuser => true}}}
2024-04-28T02:38:19.363509+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:53855, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:20.986283+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:53855, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:21.154918+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:53855, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:21.361022+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:53855, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:21.423441+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:53855, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:22.434965+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:53855, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:24.489687+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:56044 msg: authenticator_result, authenticator: password_based:built_in_database, result: {ok,#{is_superuser => true}}
2024-04-28T02:38:24.490138+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:56044 msg: authentication_result, reason: chain_result, result: {stop,{ok,#{is_superuser => true}}}
2024-04-28T02:38:24.523067+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:25.753632+00:00 [SUBSCRIBE] zhilunIotMqttGateway_test@10.1.86.88:56044 msg: subscribe, sub_id: zhilunIotMqttGateway_test, sub_opts: [nl: 0, qos: 0, rap: 0, rh: 0, sub_props: ], topic: {share,<<“group_zl_mqtt_gw”>>,<<“app/+/device/+/rx”>>}
rh => 0})])
2024-04-28T02:38:25.755251+00:00 [SUBSCRIBE] zhilunIotMqttGateway_test@10.1.86.88:56044 msg: subscribe, sub_id: zhilunIotMqttGateway_test, sub_opts: [nl: 0, qos: 1, rap: 0, rh: 0, sub_props: ], topic: {share,<<“group_zl_mqtt_gw”>>,<<“app/+/device/+/s/rx”>>}
2024-04-28T02:38:25.827091+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:25.828004+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:27.398159+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:27.549897+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:27.725063+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:27.827298+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:56044, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:29.563323+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:57723 msg: authenticator_result, authenticator: password_based:built_in_database, result: {ok,#{is_superuser => true}}
2024-04-28T02:38:29.564130+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:57723 msg: authentication_result, reason: chain_result, result: {stop,{ok,#{is_superuser => true}}}
2024-04-28T02:38:29.593467+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:30.945225+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:31.025303+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:33.306373+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:33.458189+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:33.576637+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:33.698977+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:57723, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:35.724961+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:59773 msg: authenticator_result, authenticator: password_based:built_in_database, result: {ok,#{is_superuser => true}}
2024-04-28T02:38:35.725358+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:59773 msg: authentication_result, reason: chain_result, result: {stop,{ok,#{is_superuser => true}}}
2024-04-28T02:38:35.758220+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:59773, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:36.326068+00:00 [SUBSCRIBE] zhilunIotMqttGateway_test@10.1.86.88:59773 msg: subscribe, sub_id: zhilunIotMqttGateway_test, sub_opts: [nl: 0, qos: 0, rap: 0, rh: 0, sub_props: ], topic: {share,<<“group_zl_mqtt_gw”>>,<<“app/+/device/+/rx”>>}
rh => 0})])
2024-04-28T02:38:36.327603+00:00 [SUBSCRIBE] zhilunIotMqttGateway_test@10.1.86.88:59773 msg: subscribe, sub_id: zhilunIotMqttGateway_test, sub_opts: [nl: 0, qos: 1, rap: 0, rh: 0, sub_props: ], topic: {share,<<“group_zl_mqtt_gw”>>,<<“app/+/device/+/s/rx”>>}
2024-04-28T02:38:36.328837+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:59773, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:36.329831+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:59773, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:42.104220+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:59773, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:42.104826+00:00 [SOCKET] zhilunIotMqttGateway_test@10.1.86.88:59773 msg: emqx_connection_terminated, reason: {shutdown,#{max => 1000,reason => message_queue_too_long,value => 2111}}
2024-04-28T02:38:42.105221+00:00 [info] msg: terminate, mfa: emqx_connection:terminate/2(676), peername: 10.1.86.88:59773, clientid: zhilunIotMqttGateway_test, reason: {shutdown,#{max => 1000,reason => message_queue_too_long,value => 2111}}
2024-04-28T02:38:42.717007+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:61001 msg: authenticator_result, authenticator: password_based:built_in_database, result: {ok,#{is_superuser => true}}
2024-04-28T02:38:42.717456+00:00 [AUTHN] zhilunIotMqttGateway_test@10.1.86.88:61001 msg: authentication_result, reason: chain_result, result: {stop,{ok,#{is_superuser => true}}}
2024-04-28T02:38:42.747195+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:61001, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:43.645307+00:00 [SUBSCRIBE] zhilunIotMqttGateway_test@10.1.86.88:61001 msg: subscribe, sub_id: zhilunIotMqttGateway_test, sub_opts: [nl: 0, qos: 0, rap: 0, rh: 0, sub_props: ], topic: {share,<<“group_zl_mqtt_gw”>>,<<“app/+/device/+/rx”>>}
rh => 0})])
2024-04-28T02:38:43.647064+00:00 [SUBSCRIBE] zhilunIotMqttGateway_test@10.1.86.88:61001 msg: subscribe, sub_id: zhilunIotMqttGateway_test, sub_opts: [nl: 0, qos: 1, rap: 0, rh: 0, sub_props: ], topic: {share,<<“group_zl_mqtt_gw”>>,<<“app/+/device/+/s/rx”>>}
2024-04-28T02:38:43.648184+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:61001, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:43.648992+00:00 [debug] msg: insert_channel_info, mfa: emqx_cm:insert_channel_info/3(164), peername: 10.1.86.88:61001, clientid: zhilunIotMqttGateway_test
2024-04-28T02:38:44.117719+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:61001, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}
2024-04-28T02:38:45.783001+00:00 [debug] msg: check_oom, mfa: emqx_connection:check_oom/1(1101), peername: 10.1.86.88:61001, clientid: zhilunIotMqttGateway_test, policy: #{max_heap_size => 4194304,enable => true,max_mailbox_size => 1000}

2
  1. check_oom 这个是每个进程（客户端）的隔一段时间的 debug 日志。放在这里可能有点误导，这个日志不是什么大问题。后续版本应该会考虑去掉它。

如何保证服务端处理的MQTT客户端能始终保持优先连接进行数据消费 避免在IOT设备连入过多时挤掉服务端数据处理的MQTT客户端
每个客户端在 emqx 里面都是独立的进程（可以理解为独立的线程，有独立的内存和 cpu 调度）不会互相影响。所以一般不会存在 IOT 设备过多挤掉其它的。除非你的端口数不足。
如果你想区分出这 2 种类型。可以在 dashboard 上再新建一个监听器，用不同的端口来区分分他们。这样2 都的隔离管理会更好，起停方便（虽然实际上没有什么区别）