使用k8s的operator部署的emqx集群，无法获取准确的prometheus指标值

jack · 2024 年5 月 29 日 02:30

使用docker部署单个emqx节点后，单独访问http://localhost:18083/api/v5/prometheus/stats
接口可以获取到指标值，但是为什么使用k8s的operator部署的emqx集群，在访问/api/v5/prometheus/stats接口时返回的指标值都是0

apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:
   name: emqx
   namespace: emqx
spec:
   image: emqx/emqx:5.6.0
   config: 
    data: |
      cluster {
        autoheal = on
        autoclean = 5m
      }
   listenersServiceTemplate:
     spec:
       type: LoadBalancer
   dashboardServiceTemplate:
     spec:
       type: LoadBalancer
   coreTemplate:
     metadata:
        name: emqx-core
     spec:
       replicas: 3
   replicantTemplate:
     metadata:
       name: emqx-rep
     spec:
       replicas: 3

zhongwencool · 2024 年5 月 29 日 03:24

有点奇怪。应该都是一样的才对吧。你可以在那个 pod 里面 curl 一下看看是不是对的。。。

jack · 2024 年5 月 29 日 08:36

pod内部curl的值也不对

zhongwencool · 2024 年5 月 29 日 10:24

那说明 API 没问题。可能是 emqx 的部署有问题。麻烦提供一下 emqx 启动时的完整日志看看

jack · 2024 年5 月 29 日 12:08

EMQX_API_KEY__BOOTSTRAP_FILE [api_key.bootstrap_file]: /opt/emqx/data/bootstrap_api_key
EMQX_DASHBOARD__LISTENERS__HTTP__BIND [dashboard.listeners.http.bind]: 18083
EMQX_RPC__PORT_DISCOVERY [rpc.port_discovery]: manual
EMQX_CLUSTER__DNS__RECORD_TYPE [cluster.dns.record_type]: srv
EMQX_CLUSTER__DNS__NAME [cluster.dns.name]: emqx-tc-headless.emqx58.svc.cluster.local
EMQX_CLUSTER__DISCOVERY_STRATEGY [cluster.discovery_strategy]: dns
EMQX_NODE__ROLE [node.role]: replicant
EMQX_NODE__DATA_DIR [node.data_dir]: data
EMQX_NODE__COOKIE [node.cookie]: ******
EMQX_NODE__NAME [node.name]: emqx@10.1.3.49
2024-05-29T12:02:01.672006+00:00 [info] event=server_setup_successfully driver=tcp port=5369 socket="#Port<0.7>"
2024-05-29T12:02:01.672220+00:00 [info] msg: gen_rpc_dispatcher_start
2024-05-29T12:02:01.673417+00:00 [notice] msg: Starting mria
2024-05-29T12:02:01.673937+00:00 [notice] msg: Starting mnesia
2024-05-29T12:02:01.679285+00:00 [notice] msg: Creating new mnesia schema, result: ok
2024-05-29T12:02:01.687997+00:00 [notice] msg: Starting shards
2024-05-29T12:02:01.691621+00:00 [info] msg: Setting RLOG shard config, shard: '$mria_meta_shard', tables: ['$mria_rlog_sync',mria_schema]
2024-05-29T12:02:01.691759+00:00 [info] msg: Converging schema
2024-05-29T12:02:01.692023+00:00 [info] msg: Setting RLOG shard config, shard: '$mria_meta_shard', tables: ['$mria_rlog_sync',mria_schema]
2024-05-29T12:02:01.692369+00:00 [info] msg: starting_rlog_shard, shard: '$mria_meta_shard'
2024-05-29T12:02:01.692442+00:00 [info] msg: state_change, from: disconnected, shard: '$mria_meta_shard', to: disconnected
2024-05-29T12:02:01.692745+00:00 [info] msg: Starting ekka
2024-05-29T12:02:01.693096+00:00 [info] msg: Ekka is running
2024-05-29T12:02:01.693236+00:00 [notice] msg: (re)starting_emqx_apps
2024-05-29T12:02:03.135541+00:00 [info] Mria(Membership): Node emqx@emqx-tc-core-c7c6df9cd-0.emqx-tc-headless.emqx58.svc.cluster.local up
2024-05-29T12:02:03.136053+00:00 [info] Mria(Membership): Node emqx@emqx-tc-core-c7c6df9cd-1.emqx-tc-headless.emqx58.svc.cluster.local up
2024-05-29T12:02:03.140386+00:00 [info] Mria(Membership): Node emqx@emqx-tc-core-c7c6df9cd-2.emqx-tc-headless.emqx58.svc.cluster.local up
2024-05-29T12:02:03.141303+00:00 [info] msg: mria_lb_core_discovery_new_nodes, ignored_nodes: [], node: 'emqx@10.1.3.49', previous_cores: [], returned_cores: ['emqx@emqx-tc-core-c7c6df9cd-0.emqx-tc-headless.emqx58.svc.cluster.local','emqx@emqx-tc-core-c7c6df9cd-1.emqx-tc-headless.emqx58.svc.cluster.local','emqx@emqx-tc-core-c7c6df9cd-2.emqx-tc-headless.emqx58.svc.cluster.local']
2024-05-29T12:02:03.142945+00:00 [info] msg: state_change, from: disconnected, shard: '$mria_meta_shard', to: bootstrap
2024-05-29T12:02:03.146002+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.146164+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123145, shard: '$mria_meta_shard'
2024-05-29T12:02:03.146267+00:00 [info] msg: state_change, from: bootstrap, shard: '$mria_meta_shard', to: local_replay
2024-05-29T12:02:03.146577+00:00 [info] msg: state_change, from: local_replay, shard: '$mria_meta_shard', to: normal
2024-05-29T12:02:03.146923+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: '$mria_meta_shard'
2024-05-29T12:02:03.147125+00:00 [notice] msg: Mria is running
2024-05-29T12:02:03.147196+00:00 [info] msg: Starting ekka
2024-05-29T12:02:03.147351+00:00 [info] msg: Ekka is running
2024-05-29T12:02:03.148901+00:00 [info] msg: Setting RLOG shard config, shard: emqx_common_shard, tables: [bpapi,emqx_app,emqx_banned,emqx_banned_rules,emqx_trace]
2024-05-29T12:02:03.151965+00:00 [info] msg: Setting RLOG shard config, shard: emqx_cluster_rpc_shard, tables: [cluster_rpc_commit,cluster_rpc_mfa]
2024-05-29T12:02:03.155227+00:00 [info] msg: Setting RLOG shard config, shard: emqx_cluster_rpc_shard, tables: [cluster_rpc_commit,cluster_rpc_mfa]
2024-05-29T12:02:03.157356+00:00 [info] msg: starting_rlog_shard, shard: emqx_cluster_rpc_shard
2024-05-29T12:02:03.157493+00:00 [info] msg: state_change, from: disconnected, shard: emqx_cluster_rpc_shard, to: disconnected
2024-05-29T12:02:03.159051+00:00 [info] msg: Setting RLOG shard config, shard: emqx_acl_sharded, tables: [emqx_acl]
2024-05-29T12:02:03.159309+00:00 [info] msg: state_change, from: disconnected, shard: emqx_cluster_rpc_shard, to: bootstrap
2024-05-29T12:02:03.162068+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.162182+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123161, shard: emqx_cluster_rpc_shard
2024-05-29T12:02:03.162304+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_cluster_rpc_shard, to: local_replay
2024-05-29T12:02:03.162780+00:00 [info] msg: state_change, from: local_replay, shard: emqx_cluster_rpc_shard, to: normal
2024-05-29T12:02:03.162987+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_cluster_rpc_shard
2024-05-29T12:02:03.163114+00:00 [info] msg: wait_for_cluster_rpc_shard, result: ok
2024-05-29T12:02:03.163239+00:00 [info] msg: wait_for_cluster_rpc_tables, result: ok
2024-05-29T12:02:03.163444+00:00 [info] msg: Setting RLOG shard config, shard: undefined, tables: [emqx_activated_alarm,emqx_dashboard_monitor,emqx_deactivated_alarm,emqx_delayed]
2024-05-29T12:02:03.165167+00:00 [info] msg: sync_cluster_conf_success, data_dir: data, has_deprecated_file: false, local_release: v5.6.0, remote_release: v5.6.0, synced_from_node: 'emqx@emqx-tc-core-c7c6df9cd-1.emqx-tc-headless.emqx58.svc.cluster.local', tnx_id: 0
2024-05-29T12:02:03.167489+00:00 [info] msg: Setting RLOG shard config, shard: emqx_dashboard_shard, tables: [emqx_admin,emqx_admin_jwt]
2024-05-29T12:02:03.170737+00:00 [info] msg: Setting RLOG shard config, shard: emqx_dashboard_shard, tables: [emqx_admin,emqx_admin_jwt]
2024-05-29T12:02:03.175385+00:00 [info] msg: Setting RLOG shard config, shard: emqx_common_shard, tables: [bpapi,emqx_app,emqx_banned,emqx_banned_rules,emqx_trace]
2024-05-29T12:02:03.180349+00:00 [info] msg: Setting RLOG shard config, shard: emqx_authn_shard, tables: [emqx_authn_mnesia,emqx_authn_scram_mnesia]
2024-05-29T12:02:03.183721+00:00 [info] msg: Setting RLOG shard config, shard: emqx_authn_shard, tables: [emqx_authn_mnesia,emqx_authn_scram_mnesia]
2024-05-29T12:02:03.187498+00:00 [info] msg: Setting RLOG shard config, shard: emqx_common_shard, tables: [bpapi,emqx_app,emqx_banned,emqx_banned_rules,emqx_trace]
2024-05-29T12:02:03.190845+00:00 [info] msg: Setting RLOG shard config, shard: emqx_common_shard, tables: [bpapi,emqx_app,emqx_banned,emqx_banned_rules,emqx_trace]
2024-05-29T12:02:03.193795+00:00 [info] msg: Setting RLOG shard config, shard: emqx_cm_shard, tables: [emqx_channel_registry]
2024-05-29T12:02:03.198287+00:00 [info] msg: Setting RLOG shard config, shard: undefined, tables: [emqx_activated_alarm,emqx_dashboard_monitor,emqx_deactivated_alarm,emqx_delayed]
2024-05-29T12:02:03.201110+00:00 [info] msg: Setting RLOG shard config, shard: undefined, tables: [emqx_activated_alarm,emqx_dashboard_monitor,emqx_deactivated_alarm,emqx_delayed]
2024-05-29T12:02:03.207165+00:00 [info] msg: Setting RLOG shard config, shard: undefined, tables: [emqx_activated_alarm,emqx_dashboard_monitor,emqx_deactivated_alarm,emqx_delayed]
2024-05-29T12:02:03.209866+00:00 [info] msg: Setting RLOG shard config, shard: emqx_exclusive_shard, tables: [emqx_exclusive_subscription]
2024-05-29T12:02:03.210628+00:00 [notice] alarm_handler: {clear,lc_mem_alarm}
2024-05-29T12:02:03.212884+00:00 [notice] alarm_handler: {set,{system_memory_high_watermark,[]}}
2024-05-29T12:02:03.213271+00:00 [notice] alarm_handler: {clear,lc_runq_alarm}
2024-05-29T12:02:03.214670+00:00 [info] msg: Setting RLOG shard config, shard: emqx_psk_shard, tables: [emqx_psk]
2024-05-29T12:02:03.215941+00:00 [info] msg: starting_rlog_shard, shard: emqx_common_shard
2024-05-29T12:02:03.216152+00:00 [info] msg: state_change, from: disconnected, shard: emqx_common_shard, to: disconnected
2024-05-29T12:02:03.217470+00:00 [info] msg: state_change, from: disconnected, shard: emqx_common_shard, to: bootstrap
2024-05-29T12:02:03.221042+00:00 [info] msg: Setting RLOG shard config, shard: emqx_retainer_shard, tables: [emqx_retainer_index,emqx_retainer_index_meta,emqx_retainer_message]
2024-05-29T12:02:03.225028+00:00 [info] msg: Setting RLOG shard config, shard: emqx_retainer_shard, tables: [emqx_retainer_index,emqx_retainer_index_meta,emqx_retainer_message]
2024-05-29T12:02:03.227566+00:00 [info] msg: Setting RLOG shard config, shard: emqx_retainer_shard, tables: [emqx_retainer_index,emqx_retainer_index_meta,emqx_retainer_message]
2024-05-29T12:02:03.229999+00:00 [info] msg: Setting RLOG shard config, shard: route_shard, tables: [emqx_route,emqx_route_filters,emqx_routing_node,emqx_trie]
2024-05-29T12:02:03.232056+00:00 [info] msg: Setting RLOG shard config, shard: route_shard, tables: [emqx_route,emqx_route_filters,emqx_routing_node,emqx_trie]
2024-05-29T12:02:03.234707+00:00 [info] msg: Setting RLOG shard config, shard: route_shard, tables: [emqx_route,emqx_route_filters,emqx_routing_node,emqx_trie]
2024-05-29T12:02:03.236484+00:00 [info] msg: Setting RLOG shard config, shard: emqx_shared_sub_shard, tables: [emqx_shared_subscription]
2024-05-29T12:02:03.238838+00:00 [info] msg: Setting RLOG shard config, shard: emqx_telemetry_shard, tables: [emqx_telemetry]
2024-05-29T12:02:03.241374+00:00 [info] msg: Setting RLOG shard config, shard: emqx_common_shard, tables: [bpapi,emqx_app,emqx_banned,emqx_banned_rules,emqx_trace]
2024-05-29T12:02:03.242252+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.242403+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123242, shard: emqx_common_shard
2024-05-29T12:02:03.242618+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_common_shard, to: local_replay
2024-05-29T12:02:03.243025+00:00 [info] msg: state_change, from: local_replay, shard: emqx_common_shard, to: normal
2024-05-29T12:02:03.243275+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_common_shard
2024-05-29T12:02:03.244126+00:00 [info] msg: Setting RLOG shard config, shard: route_shard, tables: [emqx_route,emqx_route_filters,emqx_routing_node,emqx_trie]
2024-05-29T12:02:03.264575+00:00 [info] msg: starting_rlog_shard, shard: route_shard
2024-05-29T12:02:03.264734+00:00 [info] msg: state_change, from: disconnected, shard: route_shard, to: disconnected
2024-05-29T12:02:03.266067+00:00 [info] msg: state_change, from: disconnected, shard: route_shard, to: bootstrap
2024-05-29T12:02:03.269860+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.270049+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123269, shard: route_shard
2024-05-29T12:02:03.270234+00:00 [info] msg: state_change, from: bootstrap, shard: route_shard, to: local_replay
2024-05-29T12:02:03.270673+00:00 [info] msg: state_change, from: local_replay, shard: route_shard, to: normal
2024-05-29T12:02:03.270948+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: route_shard
2024-05-29T12:02:03.271363+00:00 [info] msg: routing_schema_used, schema: v2
2024-05-29T12:02:03.275757+00:00 [info] msg: starting_rlog_shard, shard: emqx_exclusive_shard
2024-05-29T12:02:03.275908+00:00 [info] msg: state_change, from: disconnected, shard: emqx_exclusive_shard, to: disconnected
2024-05-29T12:02:03.276227+00:00 [info] msg: starting_rlog_shard, shard: emqx_shared_sub_shard
2024-05-29T12:02:03.276414+00:00 [info] msg: state_change, from: disconnected, shard: emqx_shared_sub_shard, to: disconnected
2024-05-29T12:02:03.277284+00:00 [info] msg: state_change, from: disconnected, shard: emqx_shared_sub_shard, to: bootstrap
2024-05-29T12:02:03.277396+00:00 [info] msg: state_change, from: disconnected, shard: emqx_exclusive_shard, to: bootstrap
2024-05-29T12:02:03.278657+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.278776+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123278, shard: emqx_shared_sub_shard
2024-05-29T12:02:03.278845+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.278913+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_shared_sub_shard, to: local_replay
2024-05-29T12:02:03.279050+00:00 [info] msg: state_change, from: local_replay, shard: emqx_shared_sub_shard, to: normal
2024-05-29T12:02:03.278969+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123278, shard: emqx_exclusive_shard
2024-05-29T12:02:03.279181+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_exclusive_shard, to: local_replay
2024-05-29T12:02:03.279244+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_shared_sub_shard
2024-05-29T12:02:03.279341+00:00 [info] msg: state_change, from: local_replay, shard: emqx_exclusive_shard, to: normal
2024-05-29T12:02:03.279796+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_exclusive_shard
2024-05-29T12:02:03.288959+00:00 [info] msg: starting_rlog_shard, shard: emqx_cm_shard
2024-05-29T12:02:03.289094+00:00 [info] msg: state_change, from: disconnected, shard: emqx_cm_shard, to: disconnected
2024-05-29T12:02:03.290518+00:00 [info] msg: state_change, from: disconnected, shard: emqx_cm_shard, to: bootstrap
2024-05-29T12:02:03.292203+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.292322+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123292, shard: emqx_cm_shard
2024-05-29T12:02:03.292529+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_cm_shard, to: local_replay
2024-05-29T12:02:03.292908+00:00 [info] msg: state_change, from: local_replay, shard: emqx_cm_shard, to: normal
2024-05-29T12:02:03.293193+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_cm_shard
Listener ssl:default on :8883 started.
Listener tcp:default on :1883 started.
Listener ws:default on :8083 started.
Listener wss:default on :8084 started.
2024-05-29T12:02:03.307775+00:00 [info] msg: wait_for_cluster_rpc_shard, result: ok
2024-05-29T12:02:03.308019+00:00 [info] msg: wait_for_cluster_rpc_tables, result: ok
2024-05-29T12:02:03.308092+00:00 [info] msg: starting_rlog_shard, shard: emqx_retainer_shard
2024-05-29T12:02:03.308202+00:00 [info] msg: state_change, from: disconnected, shard: emqx_retainer_shard, to: disconnected
2024-05-29T12:02:03.309625+00:00 [info] msg: state_change, from: disconnected, shard: emqx_retainer_shard, to: bootstrap
2024-05-29T12:02:03.312863+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.312975+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123312, shard: emqx_retainer_shard
2024-05-29T12:02:03.313076+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_retainer_shard, to: local_replay
2024-05-29T12:02:03.313208+00:00 [info] msg: state_change, from: local_replay, shard: emqx_retainer_shard, to: normal
2024-05-29T12:02:03.313518+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_retainer_shard
2024-05-29T12:02:03.346841+00:00 [info] msg: starting_rlog_shard, shard: emqx_dashboard_shard
2024-05-29T12:02:03.346998+00:00 [info] msg: state_change, from: disconnected, shard: emqx_dashboard_shard, to: disconnected
2024-05-29T12:02:03.348040+00:00 [info] msg: state_change, from: disconnected, shard: emqx_dashboard_shard, to: bootstrap
2024-05-29T12:02:03.350502+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.350641+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123350, shard: emqx_dashboard_shard
2024-05-29T12:02:03.350773+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_dashboard_shard, to: local_replay
2024-05-29T12:02:03.351021+00:00 [info] msg: state_change, from: local_replay, shard: emqx_dashboard_shard, to: normal
2024-05-29T12:02:03.351288+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_dashboard_shard
2024-05-29T12:02:03.351503+00:00 [info] msg: loading_desc, file: /opt/emqx/lib/emqx_dashboard-5.0.33/priv/desc.en.hocon
2024-05-29T12:02:03.438911+00:00 [info] msg: loading_desc, file: /opt/emqx/lib/emqx_dashboard-5.0.33/priv/desc.zh.hocon
Listener http:dashboard on :18083 started.
2024-05-29T12:02:03.533925+00:00 [info] msg: started_listener_ok, name: 'http:dashboard', pid: <0.3021.0>, port: 18083
2024-05-29T12:02:03.545496+00:00 [info] msg: authn_chains_initialization_done
2024-05-29T12:02:03.551674+00:00 [info] msg: starting_rlog_shard, shard: emqx_acl_sharded
2024-05-29T12:02:03.551796+00:00 [info] msg: state_change, from: disconnected, shard: emqx_acl_sharded, to: disconnected
2024-05-29T12:02:03.553552+00:00 [info] msg: state_change, from: disconnected, shard: emqx_acl_sharded, to: bootstrap
2024-05-29T12:02:03.555747+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.555865+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123555, shard: emqx_acl_sharded
2024-05-29T12:02:03.556016+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_acl_sharded, to: local_replay
2024-05-29T12:02:03.556200+00:00 [info] msg: state_change, from: local_replay, shard: emqx_acl_sharded, to: normal
2024-05-29T12:02:03.556493+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_acl_sharded
2024-05-29T12:02:03.558571+00:00 [info] msg: starting_rlog_shard, shard: emqx_authn_shard
2024-05-29T12:02:03.558769+00:00 [info] msg: state_change, from: disconnected, shard: emqx_authn_shard, to: disconnected
2024-05-29T12:02:03.560215+00:00 [info] msg: state_change, from: disconnected, shard: emqx_authn_shard, to: bootstrap
2024-05-29T12:02:03.562872+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.563069+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123562, shard: emqx_authn_shard
2024-05-29T12:02:03.563184+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_authn_shard, to: local_replay
2024-05-29T12:02:03.563313+00:00 [info] msg: state_change, from: local_replay, shard: emqx_authn_shard, to: normal
2024-05-29T12:02:03.563642+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_authn_shard
2024-05-29T12:02:03.577637+00:00 [info] msg: starting_rlog_shard, shard: emqx_telemetry_shard
2024-05-29T12:02:03.577816+00:00 [info] msg: state_change, from: disconnected, shard: emqx_telemetry_shard, to: disconnected
2024-05-29T12:02:03.578731+00:00 [info] msg: starting_rlog_shard, shard: emqx_psk_shard
2024-05-29T12:02:03.578948+00:00 [info] msg: state_change, from: disconnected, shard: emqx_psk_shard, to: disconnected
2024-05-29T12:02:03.579168+00:00 [info] msg: state_change, from: disconnected, shard: emqx_telemetry_shard, to: bootstrap
2024-05-29T12:02:03.580106+00:00 [info] msg: state_change, from: disconnected, shard: emqx_psk_shard, to: bootstrap
2024-05-29T12:02:03.581079+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.581163+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123580, shard: emqx_telemetry_shard
2024-05-29T12:02:03.581235+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_telemetry_shard, to: local_replay
2024-05-29T12:02:03.581441+00:00 [info] msg: state_change, from: local_replay, shard: emqx_telemetry_shard, to: normal
2024-05-29T12:02:03.581805+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_telemetry_shard
2024-05-29T12:02:03.582008+00:00 [info] msg: shard_bootstrap_complete
2024-05-29T12:02:03.582170+00:00 [info] msg: Bootstrap of the shard is complete, checkpoint: 1716984123581, shard: emqx_psk_shard
2024-05-29T12:02:03.582271+00:00 [info] msg: state_change, from: bootstrap, shard: emqx_psk_shard, to: local_replay
2024-05-29T12:02:03.582419+00:00 [info] msg: state_change, from: local_replay, shard: emqx_psk_shard, to: normal
2024-05-29T12:02:03.582745+00:00 [notice] msg: Shard fully up, node: 'emqx@10.1.3.49', shard: emqx_psk_shard
2024-05-29T12:02:03.583706+00:00 [info] msg: emqx_psk_disabled
EMQX 5.6.0 is running now!
2024-05-29T12:02:04.381161+00:00 [info] msg: regenerate_minirest_dispatch, elapsed: 846, i18n_lang: en, listeners: ['http:dashboard']

zhongwencool · 2024 年5 月 30 日 03:40

这个日志看起来启动是没有问题的。
你看的replicant节点，他还有个 core 节点，组成的集群，API 返回的都是自己节点的信息，默认并不会返回其它节点的。 5.7.0 加了参数可以返回集群的。你可以试试。
猜测你这个就是其它节点有连接有指标，但是这个 replicant 节点没有。

zhongwencool · 2024 年5 月 30 日 03:42

PS：这个也不错：GitHub - emqx/emqx-exporter: Prometheus exporter for EMQX

jack · 2024 年5 月 30 日 09:10

那我要安装这套exporter才可以查看整套集群的指标吗

zhongwencool · 2024 年5 月 30 日 09:15

不是必要的，只是exporter的指标会更全面的一点。
你要让prometheus采集所有的节点的指标，然后在他那用函数做聚合。