本机节点停止运行

EMQX版本:5.1.6
操作系统:Linux
本机节点名称:emqx0324@192.168.25.13
报错日志:
2024-03-17T10:38:40.667121+00:00 [error] message=channel_error driver=tcp socket=“#Port<0.86>” reason=“econnreset” action=stopping
2024-03-17T10:38:41.630890+00:00 [error] event=connect_to_remote_server peer="emqx0325@192.168.25.16" result=failure reason=“econnrefused”
2024-03-17T10:38:41.631716+00:00 [error] crasher: initial call: gen_rpc_client:init/1, pid: <0.24606.0>, registered_name: , exit: {{badrpc,econnrefused},[{gen_server,init_it,6,[{file,“gen_server.erl”},{line,835}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]}, ancestors: [gen_rpc_client_sup,gen_rpc_sup,<0.2129.0>], message_queue_len: 0, messages: , links: [<0.2135.0>], dictionary: , trap_exit: true, status: running, heap_size: 2586, stack_size: 28, reductions: 3567; neighbours:
2024-03-17T10:38:50.905542+00:00 [error] message=channel_error driver=tcp socket=“#Port<0.43>” reason=“econnreset” action=stopping
2024-03-17T10:38:56.918414+00:00 [warning] ‘global’ at node ‘emqx0324@192.168.25.13’ requested disconnect from node ‘emqx0325@192.168.25.16’ in order to prevent overlapping partitions
2024-03-17T10:39:04.938748+00:00 [warning] ‘global’ at node ‘emqx0324@192.168.25.13’ disconnected node ‘emqx0330@192.168.25.15’ in order to prevent overlapping partitions
2024-03-17T10:39:04.940745+00:00 [warning] ‘global’ at node ‘emqx0324@192.168.25.13’ disconnected node ‘emqx0327@192.168.25.12’ in order to prevent overlapping partitions
2024-03-17T10:39:04.941589+00:00 [warning] ‘global’ at node ‘emqx0324@192.168.25.13’ disconnected node ‘emqx0321@192.168.25.17’ in order to prevent overlapping partitions
2024-03-17T10:39:04.942019+00:00 [warning] ‘global’ at node ‘emqx0324@192.168.25.13’ disconnected node ‘emqx0331@192.168.25.14’ in order to prevent overlapping partitions
2024-03-17T10:39:04.963025+00:00 [critical] msg: Core cluster partition, mfa: mria_node_monitor:handle_info/2, line: 160, context: running_partitioned_network, from: ‘emqx0330@192.168.25.15’
2024-03-17T10:39:04.964592+00:00 [error] Mnesia(‘emqx0324@192.168.25.13’): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, ‘emqx0330@192.168.25.15’}
2024-03-17T10:39:04.967464+00:00 [warning] msg: alarm_is_activated, mfa: emqx_alarm:do_actions/3, line: 418, message: <<“Partition occurs at node emqx0330@192.168.25.15”>>, name: partition
2024-03-17T10:39:04.969629+00:00 [critical] msg: Core cluster partition, mfa: mria_node_monitor:handle_info/2, line: 160, context: running_partitioned_network, from: ‘emqx0327@192.168.25.12’
2024-03-17T10:39:04.971246+00:00 [critical] msg: Core cluster partition, mfa: mria_node_monitor:handle_info/2, line: 160, context: running_partitioned_network, from: ‘emqx0321@192.168.25.17’
2024-03-17T10:39:04.970668+00:00 [error] Mnesia(‘emqx0324@192.168.25.13’): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, ‘emqx0327@192.168.25.12’}
2024-03-17T10:39:04.974706+00:00 [error] Mnesia(‘emqx0324@192.168.25.13’): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, ‘emqx0321@192.168.25.17’}
2024-03-17T10:39:04.976050+00:00 [critical] msg: Core cluster partition, mfa: mria_node_monitor:handle_info/2, line: 160, context: running_partitioned_network, from: ‘emqx0331@192.168.25.14’
2024-03-17T10:39:04.985228+00:00 [error] Mnesia(‘emqx0324@192.168.25.13’): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, ‘emqx0331@192.168.25.14’}
2024-03-17T10:39:05.264030+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25134.0>,normal}
2024-03-17T10:39:05.264682+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25138.0>,normal}
2024-03-17T10:39:05.265224+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25139.0>,normal}
2024-03-17T10:39:05.265734+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25140.0>,normal}
2024-03-17T10:39:05.266480+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25141.0>,normal}
2024-03-17T10:39:05.267087+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25142.0>,normal}
2024-03-17T10:39:05.267973+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25143.0>,normal}
2024-03-17T10:39:05.269246+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25145.0>,normal}
2024-03-17T10:39:05.269841+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25146.0>,normal}
2024-03-17T10:39:05.270536+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25147.0>,normal}
2024-03-17T10:39:05.271109+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25148.0>,normal}
2024-03-17T10:39:05.271784+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25149.0>,normal}
2024-03-17T10:39:05.272409+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25150.0>,normal}
2024-03-17T10:39:05.273081+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25151.0>,normal}
2024-03-17T10:39:05.273603+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25152.0>,normal}
2024-03-17T10:39:05.274127+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25153.0>,normal}
2024-03-17T10:39:05.274655+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25154.0>,normal}
2024-03-17T10:39:05.275198+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25155.0>,normal}
2024-03-17T10:39:05.275728+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25156.0>,normal}
2024-03-17T10:39:05.276309+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25157.0>,normal}
2024-03-17T10:39:05.277288+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25158.0>,normal}
2024-03-17T10:39:05.277983+00:00 [error] msg: unexpected_info, mfa: emqx_router_helper:handle_info/2, line: 170, info: {‘EXIT’,<0.25159.0>,normal}

出现问题:查询本机节点集群状态(emqx_ctl cluster status)本机节点出现在了stopped_nodes中

13 这台机器的网络连不上其它机器了。出现脑裂了。可以查一下当时的网络情况。

问题1:出现这种情况的话,如果网络恢复,自身节点会从stopped_nodes中自动跳转到running_nodes吗?
问题2:如果自身连不上其他机器,自身节点会出现在stopped_nodes,这个是正常现象?
问题3:出现在stopped_nodes里面是不是意味着自身的服务器停止运行了?

  1. 不会自动变成 running, 你需要停止本节点,删除本节点下的 data/mnesia/*,然后重新启动它。启动后手动将他加入集群。
  2. 从本节点上看,stopped nodes 应该是其他节点。
  3. stopped nodes 只是说本节点连不上了,不知道是挂了还是网络断了。

问题1:当时使用emqx_ctl cluster status查询出来的结果, emqx0324@192.168.25.13就在stopped nodes中
问题2:本机节点什么情况会出现在stoppped nodes里面?