EMQX消息流入速率和流出速率异常

环境

  • EMQX 版本:
  • 操作系统版本:
    Ubuntu 22.04

重现此问题的步骤

  1. emqx崩溃重启
  2. xxx
  3. xxx

预期行为

消息流入速率和流出速率与实际一致

实际行为

消息流入速率机会没用
消息流出速率疯狂增大,100、 120、 到1000

2025-09-22T17:21:43.993786+08:00 [error] msg: connection_error, mfa: emqtt:handle_event/4(1416), error: tcp_error, reason: etimedout
2025-09-22T17:21:50.947129+08:00 [warning] msg: alarm_is_activated, mfa: emqx_alarm:do_actions/3(418), message: <<“resource down: Unknown reason”>>, name: <<“connector:mqtt:mqtt”>>
2025-09-22T17:21:50.947600+08:00 [warning] msg: health_check_failed, mfa: emqx_resource_manager:handle_connected_health_check/1(893), id: <<“connector:mqtt:mqtt”>>, status: disconnected
2025-09-22T17:21:50.947878+08:00 [warning] msg: alarm_is_activated, mfa: emqx_alarm:do_actions/3(418), message: <<“resource down: #{error => <<"Resource connector:mqtt:mqtt for channel action:mqtt:iot_rule_line:connector:mqtt:mqtt is not connected. Resource status: disconnected">>,status => disconnected}”>>, name: <<“action:mqtt:iot_rule_line:connector:mqtt:mqtt”>>
2025-09-22T17:21:50.948168+08:00 [warning] msg: alarm_is_activated, mfa: emqx_alarm:do_actions/3(418), message: <<“resource down: #{error => <<"Resource connector:mqtt:mqtt for channel action:mqtt:iot_rule_offline:connector:mqtt:mqtt is not connected. Resource status: disconnected">>,status => disconnected}”>>, name: <<“action:mqtt:iot_rule_offline:connector:mqtt:mqtt”>>
2025-09-22T17:21:50.948420+08:00 [warning] msg: alarm_is_activated, mfa: emqx_alarm:do_actions/3(418), message: <<“resource down: #{error => <<"Resource connector:mqtt:mqtt for channel action:mqtt:iot_rule_online:connector:mqtt:mqtt is not connected. Resource status: disconnected">>,status => disconnected}”>>, name: <<“action:mqtt:iot_rule_online:connector:mqtt:mqtt”>>
2025-09-22T17:22:05.984832+08:00 [warning] msg: alarm_is_deactivated, mfa: emqx_alarm:do_actions/3(424), name: <<“action:mqtt:iot_rule_line:connector:mqtt:mqtt”>>
2025-09-22T17:22:05.985244+08:00 [warning] msg: alarm_is_deactivated, mfa: emqx_alarm:do_actions/3(424), name: <<“action:mqtt:iot_rule_offline:connector:mqtt:mqtt”>>
2025-09-22T17:22:05.985492+08:00 [warning] msg: alarm_is_deactivated, mfa: emqx_alarm:do_actions/3(424), name: <<“action:mqtt:iot_rule_online:connector:mqtt:mqtt”>>
2025-09-22T17:22:05.985757+08:00 [warning] msg: alarm_is_deactivated, mfa: emqx_alarm:do_actions/3(424), name: <<“connector:mqtt:mqtt”>>

什么叫机会没用:thinking:

1对n就会这样扇型流速,(发布一条n个端订阅)。不算异常吧。
日志中的报错原因是mqtt桥接对端连接失败了

1、EMQX 流入数量几乎没有 但是流出有几百个每秒
2、实际流入数量不止每秒不到1个
3、实际流出页没有你们多

应该不会吧,
如果是这个么情况,那应该是致命bug…
麻烦提供一下可复现的步骤(代码),我们安排复现一下。

目前生产环境统计情况就是这样的,实际上我们也没有进行什么操作,不知道发生的原因是什么,只是从前段时间emqx自动重启后就出现这样的情况

也就是说重启前是正常的?
无法重现,这就很难定位了:face_exhaling:

应该说之前是正常的 然后出现异常情况导致EMQX崩溃重启,然后就异常了。而且现在实际应该还是正常的流入流出,只是EMQX上统计的数据不对。同时还有异常情况,导致规则的在线离线不正常通知了,查看依旧有比较多的异常告警。