服务器宕机后,EMQX和MySQL重启后,EMQX无法链接mysql

环境

  • EMQX 版本:5.4.1
  • 操作系统版本:centos

服务器因为内存耗资源尽导致卡死宕机,EMQX和mysql都安装在同一台机器上,加了内存条后,将服务器重庆,EMQX和MySQL也正常启动,使用navigate可以正常连接MySQL,应用程序也正常访问数据库,但是使用emqx的emqx_authn_mysql无法连接,删除原有连接重新配置依旧失败。报错日志如下:

der: id=worker_sup,pid=undefined.
2024-03-15T06:34:28.089534+00:00 [error] crasher: initial call: ecpool_worker:init/1, pid: <0.16744.0>, registered_name: , exit: {ehostunreach,[{gen_server,init_it,6,[{file,“gen_server.erl”},{line,835}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]}, ancestors: [<0.16743.0>,<0.16741.0>,ecpool_sup,<0.2760.0>], message_queue_len: 0, messages: , links: [<0.16743.0>,<0.16745.0>], dictionary: , trap_exit: true, status: running, heap_size: 376, stack_size: 28, reductions: 254; neighbours: neighbour: pid: <0.16745.0>, registered_name: , initial call: mysql_conn:init/1, current_function: {re,grun,3}, ancestors: [<0.16744.0>,<0.16743.0>,<0.16741.0>,ecpool_sup,<0.2760.0>], message_queue_len: 0, links: [<0.16744.0>], trap_exit: false, status: runnable, heap_size: 4185, stack_size: 66, reductions: 15482, current_stacktrace: [{re,urun2,3,[{file,“re.erl”},{line,769}]},{re,urun,3,[{file,“re.erl”},{line,744}]},{re,do_replace,5,[{file,“re.erl”},{line,417}]},{re,replace,4,[{file,“re.erl”},{line,387}]},{logger_formatter,format,2,[{file,“logger_formatter.erl”},{line,82}]},{logger_h_common,try_format,3,[{file,“logger_h_common.erl”},{line,407}]},{logger_h_common,do_log_to_binary,2,[{file,“logger_h_common.erl”},{line,395}]},{logger_h_common,log,2,[{file,“logger_h_common.erl”},{line,177}]},{logger_backend,call_handlers,3,[{file,“logger_backend.erl”},{line,51}]},{proc_lib,exit_p,3,[{file,“proc_lib.erl”},{line,260}]}]
2024-03-15T06:34:28.089963+00:00 [error] msg: start_ecpool_error, mfa: emqx_resource_pool:start/3(51), pool_name: <<“emqx_authn_mysql:31”>>, reason: ehostunreach
2024-03-15T06:34:28.090090+00:00 [warning] msg: start_resource_failed, mfa: emqx_resource_manager:start_resource/2(611), id: <<“emqx_authn_mysql:31”>>, reason: {start_pool_failed,<<“emqx_authn_mysql:31”>>,ehostunreach}
2024-03-15T06:34:28.090358+00:00 [warning] msg: alarm_is_activated, mfa: emqx_alarm:do_actions/3(418), message: <<“resource down: {start_pool_failed,<<"emqx_authn_mysql:31">>,ehostunreach}”>>, name: <<“emqx_authn_mysql:31”>>
2024-03-15T06:34:28.114730+00:00 [debug] msg: cluster_rpc_apply_result, mfa: emqx_cluster_rpc:log_and_alarm/3(600), entrypoint: <<“emqx:update_config/3”>>, kind: initiate, result: {ok,#{config => [#{backend => mysql,database => <<“message_center”>>,enable => true,mechanism => password_based,password => ““,password_hash_algorithm => #{name => sha256,salt_position => suffix},pool_size => 8,query => “SELECT password_hash, salt FROM mqtt_user where username = ${username} LIMIT 1”,query_timeout => 5000,server => “192.168.50.50:3306”,ssl => #{ciphers => [],depth => 10,enable => false,hibernate_after => 5000,log_level => notice,reuse_sessions => true,secure_renegotiate => true,user_lookup_fun => {fun emqx_tls_psk:lookup/3,undefined},verify => verify_peer,versions => [‘tlsv1.3’,‘tlsv1.2’]},username => <<“root”>>},#{backend => built_in_database,enable => true,mechanism => password_based,password_hash_algorithm => #{name => sha256,salt_position => suffix},user_id_type => username}],post_config_update => #{emqx_authn_config => #{enable => true,id => <<“password_based:mysql”>>,provider => emqx_authn_mysql,state => #{password_hash_algorithm => #{name => sha256,salt_position => suffix},query_timeout => 5000,resource_id => <<“emqx_authn_mysql:31”>>,tmpl_token => [{var,“username”,[<<“username”>>]}]}}},raw_config => [#{<<“backend”>> => <<“mysql”>>,<<“database”>> => <<“message_center”>>,<<“enable”>> => true,<<“mechanism”>> => <<“password_based”>>,<<“password”>> => <<””>>,<<“password_hash_algorithm”>> => #{<<“name”>> => <<“sha256”>>,<<“salt_position”>> => <<“suffix”>>},<<“pool_size”>> => 8,<<“query”>> => <<“SELECT password_hash, salt FROM mqtt_user where username = ${username} LIMIT 1”>>,<<“query_timeout”>> => <<“5s”>>,<<“server”>> => <<“192.168.50.50:3306”>>,<<“ssl”>> => #{<<“ciphers”>> => ,<<“depth”>> => 10,<<“enable”>> => false,<<“hibernate_after”>> => <<“5s”>>,<<“log_level”>> => <<“notice”>>,<<“reuse_sessions”>> => true,<<“secure_renegotiate”>> => true,<<“verify”>> => <<“verify_peer”>>,<<“versions”>> => [<<“tlsv1.3”>>,<<“tlsv1.2”>>]},<<“username”>> => <<“root”>>},#{<<“backend”>> => <<“built_in_database”>>,<<“enable”>> => true,<<“mechanism”>> => <<“password_based”>>,<<“password_hash_algorithm”>> => #{<<“name”>> => <<“sha256”>>,<<“salt_position”>> => <<“suffix”>>},<<“user_id_type”>> => <<“username”>>}]}}, tnx_id: 17


告警信息如图

这个错误的意思是,mysql 配置的地址无法访问。方便看下你的 auth_mysql 的配置么

我这边解决了,发现是服务器重启后,防火墙又被激活了,解决下防火墙的问题就可以了。