启用一个响应动作为data_to_webserver的规则后,EMQX接收消息偶尔会出现卡顿

EMQX版本 4.4 操作系统CentOS7

启用一个响应动作为data_to_webserver的规则后,接收消息偶尔会出现卡顿

如下图所示,发送客户端每隔100ms发送一条消息到EMQX,在红框框住的那段时间,EMQX停止了处理消息,大概1秒之后,以比较快的效率处理了过去1秒未处理的消息

data_to_webserver调用的服务什么都不做也存在这样的问题,应该不是data_to_webserver调用的服务卡顿导致这样的情况

data_to_webserver调用的服务与EMQX在同一台机器上,应该也不是网络波动方面的原因

把响应动作为data_to_webserver的规则禁用后,不会出现这样的卡顿

请问有什么可以排查的思路,谢谢

只看这点日志的话,没什么思路。

PS: 4.4 已经过了维护周期了。可以升级到 5.8.6 或者 5.9.0 试试。

2025-05-12T10:33:16.506716+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,54,46,55,57,50>>

2025-05-12T10:33:16.506910+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:16.792”>>)

2025-05-12T10:33:16.507041+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:16.792”>>

2025-05-12T10:33:16.607726+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,54,46,56,57,51>>

2025-05-12T10:33:16.608051+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:16.893”>>)

2025-05-12T10:33:16.608187+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:16.893”>>

2025-05-12T10:33:16.707661+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,54,46,57,57,51>>

2025-05-12T10:33:16.707819+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:16.993”>>)

2025-05-12T10:33:16.707939+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:16.993”>>

2025-05-12T10:33:16.709768+00:00 [debug] outClient@192.168.154.2:49207 file: emqx_connection.erl, line: 788, mfa: {emqx_connection,check_oom,1}, msg: check_oom, pid: <0.2060.0>, policy: #{max_heap_size => 8388608,message_queue_len => 10000}

2025-05-12T10:33:16.806254+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,48,57,49>>

2025-05-12T10:33:16.806559+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.091”>>)

2025-05-12T10:33:16.806821+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.091”>>

2025-05-12T10:33:16.908795+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,49,57,52>>

2025-05-12T10:33:16.908977+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.194”>>)

2025-05-12T10:33:16.909115+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.194”>>

2025-05-12T10:33:17.812995+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,50,57,52>>

2025-05-12T10:33:17.813294+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.294”>>)

2025-05-12T10:33:17.813504+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.294”>>

2025-05-12T10:33:17.816092+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,51,57,49>>

2025-05-12T10:33:17.816278+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.391”>>)

2025-05-12T10:33:17.816414+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.391”>>

2025-05-12T10:33:17.818365+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,52,57,51>>

2025-05-12T10:33:17.818512+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.493”>>)

2025-05-12T10:33:17.818634+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.493”>>

2025-05-12T10:33:17.820725+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,53,57,51>>

2025-05-12T10:33:17.820880+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.593”>>)

2025-05-12T10:33:17.821030+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.593”>>

2025-05-12T10:33:17.822965+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV <<48,92,0,67,57,48,51,100,48,52,101,51,45,51,102,101,54,45,52,102,48,102,45,98,55,51,51,45,102,48,102,99,50,100,57,99,100,54,97,56,47,49,53,56,49,70,53,66,75,68,50,50,49,82,48,48,50,48,48,52,57,47,100,114,111,110,101,47,111,115,100,50,48,50,53,45,48,53,45,49,50,32,49,56,58,51,51,58,49,55,46,54,57,52>>

2025-05-12T10:33:17.823162+00:00 [debug] outClient@192.168.154.2:49207 [MQTT] RECV PUBLISH(Q0, R0, D0, Topic=903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd, PacketId=undefined, Payload=<<“2025-05-12 18:33:17.694”>>)

2025-05-12T10:33:17.823303+00:00 [info] outClient@192.168.154.2:49207 PUBLISH to 903d04e3-3fe6-4f0f-b733-f0fc2d9cd6a8/1581F5BKD221R0020049/drone/osd: <<“2025-05-12 18:33:17.694”>>

在2025-05-12T10:33:06.908122+00:00之后有一个大概持续1秒的卡顿
在卡顿之前有一行check_oom的日志,不知道这个是否有影响

这里有一份详细的日志,如果可以的话麻烦再帮忙看看是否有可能能解决这个问题,谢谢

通过网盘分享的文件:1.log
链接: 百度网盘 请输入提取码 提取码: 2rzs

这个版本太老了(过了维护周期了),用来调试工具都没有。
如果实在不能升级,可以直接联系 EMQX 专家 1v1 支持

好的,尝试了5.9的新版,没有类似的问题,谢谢。