emqx5.3.2 只建了webhook转发消息,会出现客户端连接断开的问题

在服务器上建立了emqx单机服务,只建立了webhook进行数据转发,启动以后在登录admin账号的情况下是好的,但是退出登录以后一段时间就会发现客户端断开了,登录以后重新连接了,但是如果太久没有登录,就完全断开了,登录重连以后,数据也不会再进行转发,只有关闭webhook再重开,又会恢复,恢复的时候仅有的两个客户端还不会全恢复,需要多重开几次才能都恢复

环境:
版本:emqx5.3.2
操作系统:centos7

报错信息
重开webhook前的报错
2024-12-17T12:47:04.111042+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:04.265979+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:33.436959+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:35.693043+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:47.523031+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:48.293974+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:48.487975+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:48.651999+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:49.288986+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:49.358972+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:49.612961+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}
2024-12-17T12:47:49.767952+08:00 [error] msg: send_error, mfa: emqx_resource_buffer_worker:handle_query_result_pure/3(941), id: <<“action:http:my_webhook_WH_D:connector:http:connector_my_webhook_WH_D”>>, reason: {recoverable_error,timeout}

重开webhook后的报错
2024-12-17T12:47:58.738849+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:13757, clientid: weining, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,killed}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,713}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]
2024-12-17T12:47:58.804688+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:46524, clientid: bailidujuan, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,killed}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,713}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]
2024-12-17T12:54:38.922095+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:58532, clientid: weining, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,killed}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,713}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]
2024-12-17T12:57:26.116683+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:10624, clientid: weining, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,killed}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,713}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]
2024-12-17T12:57:26.119152+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:1270, clientid: bailidujuan, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,killed}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,709}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]
2024-12-17T20:25:16.494330+08:00 [error] msg: session_stepdown_request_exception, mfa: emqx_cm:request_stepdown/3(443), peername: xxx.xxx.xxx.xxx:32394, clientid: bailidujuan, action: {takeover,‘begin’}, reason: {killed,{gen_server,call,[<0.16534.1407>,{takeover,‘begin’},15000]}}, stacktrace: [{gen_server,call,3,[{file,“gen_server.erl”},{line,385}]},{emqx_cm,request_stepdown,3,[{file,“emqx_cm.erl”},{line,414}]},{emqx_cm,do_takeover_begin,2,[{file,“emqx_cm.erl”},{line,369}]},{emqx_cm,takeover_session,2,[{file,“emqx_cm.erl”},{line,350}]},{emqx_cm,takeover_session_begin,2,[{file,“emqx_cm.erl”},{line,300}]},{emqx_session_mem,open,2,[{file,“emqx_session_mem.erl”},{line,201}]},{emqx_session,open,2,[{file,“emqx_session.erl”},{line,209}]},{emqx_cm,‘-open_session/3-fun-1-’,5,[{file,“emqx_cm.erl”},{line,278}]},{emqx_cm_locker,trans,3,[{file,“emqx_cm_locker.erl”},{line,51}]},{emqx_channel,process_connect,2,[{file,“emqx_channel.erl”},{line,596}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}], stale_channel: undefined, stale_pid: <0.16534.1407>
2024-12-18T10:48:48.711216+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:13669, clientid: weining, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,killed}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,709}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]
2024-12-18T10:48:48.711922+08:00 [error] msg: action_failed, mfa: emqx_rule_runtime:handle_action/4(343), peername: xxx.xxx.xxx.xxx:13669, clientid: weining, action: {bridge_v2,http,my_webhook_WH_D}, exception: error, reason: {worker_down,noproc}, stacktrace: [{emqx_resource_buffer_worker,pick_call,4,[{file,“emqx_resource_buffer_worker.erl”},{line,369}]},{emqx_rule_runtime,do_handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,377}]},{emqx_rule_runtime,handle_action,4,[{file,“emqx_rule_runtime.erl”},{line,326}]},{emqx_rule_runtime,‘-handle_action_list/4-lc$^0/1-0-’,4,[{file,“emqx_rule_runtime.erl”},{line,321}]},{emqx_rule_runtime,do_apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,180}]},{emqx_rule_runtime,apply_rule,3,[{file,“emqx_rule_runtime.erl”},{line,72}]},{emqx_rule_runtime,apply_rule_discard_result,3,[{file,“emqx_rule_runtime.erl”},{line,65}]},{emqx_rule_runtime,apply_rules,3,[{file,“emqx_rule_runtime.erl”},{line,61}]},{emqx_rule_events,on_message_publish,2,[{file,“emqx_rule_events.erl”},{line,142}]},{emqx_hooks,safe_execute,2,[{file,“emqx_hooks.erl”},{line,205}]},{emqx_hooks,do_run_fold,3,[{file,“emqx_hooks.erl”},{line,185}]},{emqx_broker,publish,1,[{file,“emqx_broker.erl”},{line,219}]},{emqx_channel,do_publish,3,[{file,“emqx_channel.erl”},{line,709}]},{emqx_connection,with_channel,3,[{file,“emqx_connection.erl”},{line,840}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,493}]},{emqx_connection,process_msg,2,[{file,“emqx_connection.erl”},{line,499}]},{emqx_connection,handle_recv,3,[{file,“emqx_connection.erl”},{line,455}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]

麻烦帮忙看看是哪里的问题

这个是你重新启动 http server 了?还是只是在 emqx 的后台关停 webhook 后再开起来?

登录哪里?你的 http server,还是你的客户端,还是 emqx 的后台?
这个日志的意思是:

这个 http server 连不上。

这个是你的客户端一直连不上的直接原因。他一直卡在session重连中。但是这个日志还是找不到为什么会卡。麻烦详细描述一下你的整个流程是怎么样的。

谢谢回复,我是放着不动一段时间,没有重启服务器,登录和退出的都是emqx的后台管理,我整个的操作过程就是先登录管理后台,就会发现客户端的连接时间就是登录的那个时候,今天就是截图上那个时间登录的,然后这个时候点客户端进去的publish,qos0,qos1,qos2的包全部都是0,测试是否有数据转发出来发现并没有数据出来,然后去webhook那里重启一下就会发现publish那里有数据了,测试转发也发现有数据了,但是我有两个客户端,重启以后一般两个客户端会有一个有数据了,多重启几下以后才会两个一起都有了



试试马上就要发布的 5.8.4