K8S EMQX运行一段时间突发故障,看容器状态是RunContainerError 故障节点为数据节点 复制节点工作正常 进数据节点看输出日志
2025-10-17T01:14:34.739579+00:00 [error] crasher: initial call: emqx_delayed:init/1, pid: <0.2990.0>, registered_name: emqx_delayed, error: {badarg,[{ets,internal_delete_all,[emqx_delayed,undefined],[{error_info,#{cause => id,module => erl_stdlib_errors}}]},{ets,delete_all_objects,1,[{file,“ets.erl”},{line,126}]},{emqx_delayed,do_load_or_unload,2,[{file,“emqx_delayed.erl”},{line,420}]},{gen_server,try_terminate,3,[{file,“gen_server.erl”},{line,1161}]},{gen_server,terminate,10,[{file,“gen_server.erl”},{line,1351}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]}, ancestors: [emqx_modules_sup,<0.2975.0>], message_queue_len: 0, messages: , links: [<0.2976.0>], dictionary: , trap_exit: true, status: running, heap_size: 6772, stack_size: 28, reductions: 17097597; neighbours:
2025-10-17T01:14:34.739994+00:00 [error] Supervisor: {local,emqx_modules_sup}. Context: shutdown_error. Reason: {badarg,[{ets,internal_delete_all,[emqx_delayed,undefined],[{error_info,#{cause => id,module => erl_stdlib_errors}}]},{ets,delete_all_objects,1,[{file,“ets.erl”},{line,126}]},{emqx_delayed,do_load_or_unload,2,[{file,“emqx_delayed.erl”},{line,420}]},{gen_server,try_terminate,3,[{file,“gen_server.erl”},{line,1161}]},{gen_server,terminate,10,[{file,“gen_server.erl”},{line,1351}]},{proc_lib,wake_up,3,[{file,“proc_lib.erl”},{line,250}]}]}. Offender: id=emqx_delayed,pid=<0.2990.0>.
2025-10-17T01:14:34.743028+00:00 [info] msg: stopping_mysql_connector, mfa: emqx_mysql:on_stop/2, line: 147, connector: <<“emqx_authn_mysql:104”>>
2025-10-17T01:14:34.744726+00:00 [info] msg: stop_ecpool_ok, mfa: emqx_resource_pool:stop/1, line: 58, pool_name: <<“emqx_authn_mysql:104”>>
2025-10-17T01:14:34.744914+00:00 [error] msg: emqx_alarm_safe_call_exception, mfa: emqx_alarm:safe_call/1, line: 492, reason: {noproc,{gen_server,call,[emqx_alarm,{deactivate_alarm,<<“emqx_authn_mysql:104”>>,no_details,<<>>}]}}, stacktrace: [{gen_server,call,2,[{file,“gen_server.erl”},{line,370}]},{emqx_alarm,safe_call,1,[{file,“emqx_alarm.erl”},{line,482}]},{emqx_resource_manager,stop_resource,1,[{file,“emqx_resource_manager.erl”},{line,518}]},{emqx_resource_manager,terminate,3,[{file,“emqx_resource_manager.erl”},{line,331}]},{gen_statem,terminate,7,[{file,“gen_statem.erl”},{line,2602}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]
{“Kernel pid terminated”,application_controller,“{application_terminated,emqx,shutdown}”}
Kernel pid terminated (application_controller) ({application_terminated,emqx,shutdown})
Crash dump is being written to: /opt/emqx/log/erl_crash.dump…done
经过对挂载目录里mnesia清理 重建容器后恢复正常 疑似是一个bug 还是什么问题会影响到 升级版本对这个问题会有一定优化修复么