EMQX4.3.3 web_hook 进行数据推送插入TDengine 出现问题,回退4.2.3 不会报错

环境信息

你好,按照此博客示例进行搭建<EMQ X+TDengine 搭建 MQTT 物联网可视化平台 >

服务器: Ubuntu20.04 docker环境部署

TDengine: 2.1.2.0

EMQX: 4.3.3

问题描述

按照博客中 mock.js 进行模拟数据推送,在EMQX 配置相关规则转换成sql后,使用 http://127.0.0.1:6041/rest/sql 进行数据插入,频率5s,出现部分成功,部分失败问题;

当mock.js 中模拟emqx 模拟客户端数设置为 5时,emqx 日志没有报错;
当模拟客户端数大于6时。报错概率增大;
当回退EMQX 版本到4.2.3 后模拟客户端数量设置为100 也没有出现相关错误。

4.3.3版本错误日志如下:


2021-06-30T16:09:41.931559+08:00 [error] mock_client_9@192.168.180.155:53754 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:41.931218+08:00 [warning] mock_client_1@192.168.180.155:53738 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:41.932013+08:00 [warning] mock_client_4@192.168.180.155:53744 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:41.932060+08:00 [error] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:41.932033+08:00 [error] mock_client_8@192.168.180.155:53752 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:41.935464+08:00 [warning] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:41.935460+08:00 [error] mock_client_8@192.168.180.155:53752 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:47.049316+08:00 [error] mock_client_8@192.168.180.155:53752 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:47.049349+08:00 [error] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:47.049398+08:00 [warning] mock_client_4@192.168.180.155:53744 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:47.163105+08:00 [warning] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:47.163169+08:00 [error] mock_client_4@192.168.180.155:53744 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:51.949099+08:00 [warning] mock_client_8@192.168.180.155:53752 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:51.949181+08:00 [error] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:51.950261+08:00 [warning] mock_client_5@192.168.180.155:53746 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:51.950279+08:00 [error] mock_client_9@192.168.180.155:53754 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:57.055779+08:00 [error] mock_client_8@192.168.180.155:53752 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:57.055789+08:00 [error] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:57.055723+08:00 [warning] mock_client_4@192.168.180.155:53744 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:57.058228+08:00 [error] mock_client_7@192.168.180.155:53750 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:57.058443+08:00 [warning] mock_client_1@192.168.180.155:53738 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:57.058597+08:00 [warning] mock_client_5@192.168.180.155:53746 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:57.058734+08:00 [error] mock_client_9@192.168.180.155:53754 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:09:57.167098+08:00 [warning] mock_client_6@192.168.180.155:53748 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:09:57.167303+08:00 [error] mock_client_4@192.168.180.155:53744 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:34:15.217833+08:00 [error] mock_client_6@192.168.180.155:53806 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:34:15.217759+08:00 [warning] mock_client_4@192.168.180.155:53802 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:34:25.227209+08:00 [warning] mock_client_6@192.168.180.155:53806 [WebHook Action] HTTP request failed with status code: 507
2021-06-30T16:34:25.227492+08:00 [error] mock_client_4@192.168.180.155:53802 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:34:30.671415+08:00 [error] mock_client_4@192.168.180.155:53802 [WebHook Action] HTTP request error: {closed,"The connection was lost."}
2021-06-30T16:34:30.671532+08:00 [warning] mock_client_6@192.168.180.155:53806 [WebHook Action] HTTP request failed with status code: 507


企业版4.3.3暂时还未开发,不知道你的版本是?

错误日志提示HTTP链接被拒绝,并出现大量的507错误。看下你的网络配置是否有问题

你好,上面环境EMQX版本如下

  1. EMQX 版本是 Broker 4.3.3
  2. 网络配置是正常的,上面提到,当EMQX 客户端数小的时候出现错误概率小,所以是有部分成功,有部分失败。如果是网络配置有问题的话应该全部失败吧
  3. 回退到EMQX4.2.3 不会出现类似问题
  4. 我是按照 EMQ X+TDengine 搭建 MQTT 物联网可视化平台 进行环境搭建测试的,如果方便的话,用docker 快速部署可以复现问题,之前在群里也请教过,那边没解决,让我过来这边提问的