问题：为什么ssl客户端无法连接ExProto中配置的ssl监听器？目前ExProto中对ssl接入有什么特殊的要求吗？我要怎么配置才能让ssl客户端接入到网关ExProto中呢呢？
EMQX服务：docker容器
EMQX版本：emqx-5.3.2
具体操作如下：
-
通过EMQX的前端，使能并配置网关ExProto，并使用emqx-extension-examples/exproto-svr-python作为gRPC服务；
-
网关ExProto默认添加了一个TCP监听器，监听端口7993，我用python写了一个tcp客户端连接7993端口并发送数据，gRPC服务能正常响应；
-
按照https://docs.emqx.com/zh/enterprise/latest/network/tls-certificate.html生成了一套证书，并将其配置为MQTT的SSL证书（新增的8885端口）并验证通过。
mqttx sub -t ‘test/1’ -h server -p 8885 --protocol mqtts --ca ./test1/rootCA.crt --cert ./test1/client.crt --key ./test1/client.key
[1/6/2024] [10:32:11 PM] › … Connecting…
[1/6/2024] [10:32:11 PM] ›
Connected
[1/6/2024] [10:32:11 PM] › … Subscribing to test/1…
[1/6/2024] [10:32:11 PM] › Subscribed to test/1
[1/6/2024] [10:32:15 PM] › payload: hello
mqttx pub -t ‘test/1’ -m “hello” -h server -p 8885 --protocol mqtts --ca ./test1/rootCA.crt --cert ./test1/client.crt --key ./test1/client.key
[1/6/2024] [10:32:15 PM] › … Connecting…
[1/6/2024] [10:32:15 PM] › Connected
[1/6/2024] [10:32:15 PM] › … Message publishing…
[1/6/2024] [10:32:15 PM] › Message published
4.在网关ExProto中添加了一个SSL监听器，监听端口7891，我用python写了一个SSL客户端连接7891端口,提示认证错误：
[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)
其中SSL客户端程序代码片段：
while True:
# 创建原始套接字
tcp_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# 创建SSL上下文
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.verify_mode = ssl.CERT_REQUIRED
# 加载证书和私钥
context.load_cert_chain(args.cert, args.key)
context.load_verify_locations(args.ca)
# 创建SSL套接字
ssl_sock = context.wrap_socket(tcp_sock, server_hostname=args.hostname)
# 连接SSL服务器
ssl_sock.connect((args.host, args.port))
# 发送和接收数据
while True:
data = 'Hello, server!'
ssl_sock.send(data.encode())
print(f"{txn}. ==> {data}")
txn = txn + 1
data = ssl_sock.recv(1024)
print(f"{rxn}. <== {data.decode()}")
rxn = rxn + 1
time.sleep(1)
except Exception as e:
print(e)
if not args.try_again:
ssl_sock.close()
break
else:
time.sleep(1)
# 关闭连接
ssl_sock.close()
- 使用openssl检查端口7891，输出日志如下：
openssl s_client -servername server -cert ./test1/client.crt -key ./test1/client.key -CAfile ./test1/rootCA.crt -connect 192.168.31.67:7891
CONNECTED(00000003)
00446190CD7F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:ssl/record/rec_layer_s3.c:304:
no peer certificate available
No client certificate CA names sent
SSL handshake has read 0 bytes and written 319 bytes
Verification: OK
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
6.使用sudo docker logs -f -n 100 emqx0查看5中操作对应emqx的日志，内容如下：
2024-01-07T08:17:08.691973+00:00 [error] State machine <0.26940.0> terminating. Reason: function_clause. Stack: [{public_key,pkix_decode_cert,[,otp],[{file,“public_key.erl”},{line,516}]},{ssl_handshake,get_cert_params,1,[{file,“ssl_handshake.erl”},{line,1764}]},{tls_handshake_1_3,get_certificate_params,1,[{file,“tls_handshake_1_3.erl”},{line,2450}]},{tls_handshake_1_3,select_server_cert_key_pair,7,[{file,“tls_handshake_1_3.erl”},{line,3013}]},{tls_handshake_1_3,do_start,2,[{file,“tls_handshake_1_3.erl”},{line,678}]},{tls_connection_1_3,do_server_start,2,[{file,“tls_connection_1_3.erl”},{line,578}]},{gen_statem,loop_state_callback,11,[{file,“gen_statem.erl”},{line,1426}]},{tls_connection,init,1,[{file,“tls_connection.erl”},{line,160}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]. Last event: {internal,{client_hello,{3,3},<<90,216,191,7,222,3,95,1,18,213,42,215,142,210,151,111,134,251,74,227,220,82,61,201,202,17,90,162,169,215,190,111>>,<<139,102,177,63,74,213,244,78,219,173,8,34,217,219,239,1,168,73,188,221,11,2,157,59,112,219,30,79,114,9,178,186>>,undefined,[<<19,2>>,<<19,3>>,<<19,1>>,<<“�,”>>,<<“�0”>>,<<0,159>>,<<204,169>>,<<204,168>>,<<204,170>>,<<“�+”>>,<<“�/”>>,<<0,158>>,<<“�$”>>,<<“�(”>>,<<0,107>>,<<“�#”>>,<<“�’”>>,<<0,103>>,<<“�\n”>>,<<192,20>>,<<0,57>>,<<“�\t”>>,<<192,19>>,<<0,51>>,<<0,157>>,<<0,156>>,<<0,61>>,<<0,60>>,<<0,53>>,<<0,47>>,<<0,255>>],[0],#{alpn => undefined,certificate_authorities => undefined,client_hello_versions => {client_hello_versions,[{3,4},{3,3},{3,2},{3,1}]},cookie => undefined,ec_point_formats => {ec_point_formats,[0,1,2]},elliptic_curves => {supported_groups,[x25519,secp256r1,x448,secp521r1,secp384r1,ffdhe2048,ffdhe3072,ffdhe4096,ffdhe6144,ffdhe8192]},key_share => {key_share_client_hello,[{key_share_entry,x25519,<<194,126,201,60,131,115,79,76,131,33,230,58,148,197,184,72,202,113,107,170,102,34,13,37,116,124,26,17,238,54,115,35>>}]},pre_shared_key => undefined,psk_key_exchange_modes => {psk_key_exchange_modes,[psk_dhe_ke]},signature_algs => {signature_algorithms,[ecdsa_secp256r1_sha256,ecdsa_secp384r1_sha384,ecdsa_secp521r1_sha512,eddsa_ed25519,eddsa_ed448,rsa_pss_pss_sha256,rsa_pss_pss_sha384,rsa_pss_pss_sha512,rsa_pss_rsae_sha256,rsa_pss_rsae_sha384,rsa_pss_rsae_sha512,rsa_pkcs1_sha256,rsa_pkcs1_sha384,rsa_pkcs1_sha512,{sha224,ecdsa},{sha224,rsa},{sha224,dsa},{sha256,dsa},{sha384,dsa},{sha512,dsa}]},signature_algs_cert => undefined,sni => {sni,“server”}}}}. State: [{data,[{“State”,{start,{state,{static_env,server,gen_tcp,tls_gen_connection,tcp,tcp_closed,tcp_error,tcp_passive,“localhost”,7891,#Port<0.2963>,#Ref<0.1557852878.999686148.228598>,no_longer_defined,ssl_server_session_cache_db,{ssl_crl_cache,{{#Ref<0.1557852878.999686148.228601>,#Ref<0.1557852878.999686148.228602>},}},{#Ref<0.1557852878.999686148.228599>,#Ref<0.1557852878.999686148.228600>},#Ref<0.1557852878.1001390082.109200>,[{session_id_tracker,<0.3150.0>}]},“",#{sender_spawn_opts => [],dhfile => undefined,anti_replay => undefined,ocsp_responder_certs => [],psk_identity => "”,honor_cipher_order => false,certfile => <<>>,dh => “",server_name_indication => undefined,verify_fun => {#Fun<ssl.12.97316091>,[]},verify => verify_none,key_update_at => 388736063997,sni_hosts => [],use_ticket => undefined,receiver_spawn_opts => [],handshake => full,hibernate_after => infinity,ciphers => [<<19,2>>,<<19,1>>,<<19,3>>,<<19,4>>,<<19,5>>,<<“�,”>>,<<“�0”>>,<<“�$”>>,<<“�(”>>,<<“�.”>>,<<“�2”>>,<<“�&”>>,<<"�">>,<<0,163>>,<<0,106>>,<<0,157>>,<<0,61>>,<<“�+”>>,<<“�/”>>,<<“�#”>>,<<“�’”>>,<<“�-”>>,<<“�1”>>,<<“�%”>>,<<“�)”>>,<<0,162>>,<<0,64>>,<<0,156>>,<<0,60>>,<<“�\n”>>,<<192,20>>,<<0,56>>,<<192,5>>,<<192,15>>,<<“�\t”>>,<<192,19>>,<<0,50>>,<<192,4>>,<<192,14>>,<<0,173>>,<<0,183>>,<<0,172>>,<<0,182>>,<<0,149>>,<<0,148>>],beast_mitigation => one_n_minus_one,log_level => notice,secure_renegotiate => true,keyfile => <<>>,cacerts => "",ocsp_nonce => true,certificate_status => undefined,fail_if_no_peer_cert => false,honor_ecc_order => false,reuse_session => #Fun<ssl.13.97316091>,middlebox_comp_mode => true,alpn_advertised_protocols => undefined,max_handshake_size => 262144,next_protocol_selector => undefined,reuse_sessions => true,protocol => tls,cookie => true,crl_check => false,fallback => undefined,password => "",erl_dist => false,key => "",alpn_preferred_protocols => undefined,customize_hostname_check => [],next_protocols_advertised => undefined,certificate_authorities => true,partial_chain => #Fun<ssl.11.97316091>,supported_groups => {supported_groups,[x25519,x448,secp256r1,secp384r1]},session_tickets => disabled,ocsp_stapling => false,early_data => undefined,depth => 10,signature_algs => [eddsa_ed25519,eddsa_ed448,ecdsa_secp521r1_sha512,ecdsa_secp384r1_sha384,ecdsa_secp256r1_sha256,rsa_pss_pss_sha512,rsa_pss_pss_sha384,rsa_pss_pss_sha256,rsa_pss_rsae_sha512,rsa_pss_rsae_sha384,rsa_pss_rsae_sha256,{sha512,ecdsa},{sha512,rsa},{sha384,ecdsa},{sha384,rsa},{sha256,ecdsa},{sha256,rsa},{sha224,ecdsa},{sha224,rsa},{sha,ecdsa},{sha,rsa},{sha,dsa}],crl_cache => {ssl_crl_cache,{internal,[]}},renegotiate_at => 268435456,signature_algs_cert => undefined,sni_fun => undefined,user_lookup_fun => undefined,max_fragment_length => undefined,padding_check => true,client_renegotiation => true,cacertfile => <<>>,cert => "",keep_secrets => false,eccs => {elliptic_curves,[{1,3,132,0,39},{1,3,132,0,38},{1,3,132,0,35},{1,3,36,3,3,2,8,1,1,13},{1,3,132,0,36},{1,3,132,0,37},{1,3,36,3,3,2,8,1,1,11},{1,3,132,0,34},{1,3,132,0,16},{1,3,132,0,17},{1,3,36,3,3,2,8,1,1,7},{1,3,132,0,10},{1,2,840,10045,3,1,7},{1,3,132,0,3},{1,3,132,0,26},{1,3,132,0,27},{1,3,132,0,32},{1,3,132,0,33},{1,3,132,0,24},{1,3,132,0,25},{1,3,132,0,31},{1,2,840,10045,3,1,1},{1,3,132,0,1},{1,3,132,0,2},{1,3,132,0,15},{1,3,132,0,9},{1,3,132,0,8},{1,3,132,0,30}]},srp_identity => "“,versions => [{3,4},{3,3},{3,2},{3,1}]},{socket_options,binary,0,0,0,false},”“,”“,not_requested,#{active_n => 100,active_n_toggle => false,change_cipher_spec => ignore,sender => <0.26939.0>},”“,undefined,”“,”“,”*”,undefined,{<0.26936.0>,#Ref<0.1557852878.1001914369.216528>},undefined}}}]}].
2024-01-07T08:17:08.692869+00:00 [error] crasher: initial call: ssl_gen_statem:init/1, pid: <0.26940.0>, registered_name: , error: {function_clause,[{public_key,pkix_decode_cert,[,otp],[{file,“public_key.erl”},{line,516}]},{ssl_handshake,get_cert_params,1,[{file,“ssl_handshake.erl”},{line,1764}]},{tls_handshake_1_3,get_certificate_params,1,[{file,“tls_handshake_1_3.erl”},{line,2450}]},{tls_handshake_1_3,select_server_cert_key_pair,7,[{file,“tls_handshake_1_3.erl”},{line,3013}]},{tls_handshake_1_3,do_start,2,[{file,“tls_handshake_1_3.erl”},{line,678}]},{tls_connection_1_3,do_server_start,2,[{file,“tls_connection_1_3.erl”},{line,578}]},{gen_statem,loop_state_callback,11,[{file,“gen_statem.erl”},{line,1426}]},{tls_connection,init,1,[{file,“tls_connection.erl”},{line,160}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]}, ancestors: [<0.26938.0>,tls_connection_sup,tls_sup,ssl_connection_sup,ssl_sup,<0.2131.0>], message_queue_len: 0, messages: , links: [<0.26938.0>], dictionary: [{log_level,notice},{ssl_pem_cache,ssl_pem_cache},{ssl_manager,ssl_manager}], trap_exit: true, status: running, heap_size: 46422, stack_size: 28, reductions: 47767; neighbours:
2024-01-07T08:17:08.693302+00:00 [error] Supervisor: {<0.26938.0>,tls_dyn_connection_sup}. Context: child_terminated. Reason: {function_clause,[{public_key,pkix_decode_cert,[,otp],[{file,“public_key.erl”},{line,516}]},{ssl_handshake,get_cert_params,1,[{file,“ssl_handshake.erl”},{line,1764}]},{tls_handshake_1_3,get_certificate_params,1,[{file,“tls_handshake_1_3.erl”},{line,2450}]},{tls_handshake_1_3,select_server_cert_key_pair,7,[{file,“tls_handshake_1_3.erl”},{line,3013}]},{tls_handshake_1_3,do_start,2,[{file,“tls_handshake_1_3.erl”},{line,678}]},{tls_connection_1_3,do_server_start,2,[{file,“tls_connection_1_3.erl”},{line,578}]},{gen_statem,loop_state_callback,11,[{file,“gen_statem.erl”},{line,1426}]},{tls_connection,init,1,[{file,“tls_connection.erl”},{line,160}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]}. Offender: id=receiver,pid=<0.26940.0>.
2024-01-07T08:17:08.693310+00:00 [error] supervisor: {esockd_connection_sup,<0.26936.0>}, errorContext: connection_shutdown, reason: {ssl_failure,{{function_clause,[{public_key,pkix_decode_cert,[,otp],[{file,“public_key.erl”},{line,516}]},{ssl_handshake,get_cert_params,1,[{file,“ssl_handshake.erl”},{line,1764}]},{tls_handshake_1_3,get_certificate_params,1,[{file,“tls_handshake_1_3.erl”},{line,2450}]},{tls_handshake_1_3,select_server_cert_key_pair,7,[{file,“tls_handshake_1_3.erl”},{line,3013}]},{tls_handshake_1_3,do_start,2,[{file,“tls_handshake_1_3.erl”},{line,678}]},{tls_connection_1_3,do_server_start,2,[{file,“tls_connection_1_3.erl”},{line,578}]},{gen_statem,loop_state_callback,11,[{file,“gen_statem.erl”},{line,1426}]},{tls_connection,init,1,[{file,“tls_connection.erl”},{line,160}]},{proc_lib,init_p_do_apply,3,[{file,“proc_lib.erl”},{line,240}]}]},{gen_statem,call,[<0.26940.0>,{start,15000},infinity]}}}, offender: [{pid,<0.26936.0>},{name,connection},{mfargs,{emqx_gateway_conn,start_link,[#{chann_mod => emqx_exproto_channel,ctx => #{cm => <0.3769.0>,gwname => exproto},enable => true,enable_authn => true,enable_stats => true,frame_mod => emqx_exproto_frame,grpc_client_channel => exproto,grpc_client_service_name => ‘ConnectionUnaryHandler’,handler => exproto,idle_timeout => 30000,listener => {exproto,ssl,‘ssl-7891’},mountpoint => <<>>}]}}]
