security.tls.v*_*_server & security.tls.v*_*_client Errors & Module timeout exceeded

[duncangreene]

Describe the bug
Using 2.1.1 I'm getting the below on a DUT that does not offer TLS on port 443. In fact port 443 is closed.

Using 2.1 we were getting the below correct 'Feature Not Detected' for the same DUT (the additional 'required result' column is present because this excerpt has come from a regenerated .zip download).

To Reproduce
Steps to reproduce the behavior:

1. Run 2.1.1 against a DUT that doesn't offer any support for TLS (on port 443).
2. See error.

Expected behavior
TLS module should be reporting 'Feature Not Detected' as applicable.

Error logs
2.1.1.zip
2.1.zip

[duncangreene]

This issue is being caused by a TLS module timeout.

In the working 2.1 against this DUT, the TLS module finishes ~3 minutes after module start.

Jan 06 11:57:18 test_module INFO    Starting TLS Module
Jan 06 11:57:20 test_tls INFO    Resolved device IP: 10.10.10.14
Jan 06 11:57:20 test_tls INFO    Running security.tls.v1_0_client
Jan 06 11:57:20 test_tls INFO    Validating client for TLS: 1.0
Jan 06 11:57:31 test_tls INFO    Checking client for TLS Protocol client connections
Jan 06 11:57:36 test_tls INFO    No client hello packets detected
Jan 06 11:57:36 test_tls INFO    Checking client for non-TLS client connections
Jan 06 11:57:42 test_tls INFO    Checking client for TLS client connections
Jan 06 11:57:47 test_tls INFO    Checking client for unsupported TLS client connections
Jan 06 11:57:47 test_tls INFO    Validating client for TLS: 1.1
Jan 06 11:57:52 test_tls INFO    Checking client for TLS Protocol client connections
Jan 06 11:57:57 test_tls INFO    No client hello packets detected
Jan 06 11:57:57 test_tls INFO    Checking client for non-TLS client connections
Jan 06 11:58:02 test_tls INFO    Checking client for TLS client connections
Jan 06 11:58:07 test_tls INFO    Checking client for unsupported TLS client connections
Jan 06 11:58:07 test_tls INFO    Validating client for TLS: 1.2
Jan 06 11:58:12 test_tls INFO    Checking client for TLS Protocol client connections
Jan 06 11:58:17 test_tls INFO    No client hello packets detected
Jan 06 11:58:17 test_tls INFO    Checking client for non-TLS client connections
Jan 06 11:58:22 test_tls INFO    Checking client for TLS client connections
Jan 06 11:58:27 test_tls INFO    Checking client for unsupported TLS client connections
Jan 06 11:58:27 test_tls INFO    Validating client for TLS: 1.3
Jan 06 11:58:32 test_tls INFO    Checking client for TLS Protocol client connections
Jan 06 11:58:37 test_tls INFO    No client hello packets detected
Jan 06 11:58:37 test_tls INFO    Checking client for non-TLS client connections
Jan 06 11:58:42 test_tls INFO    Checking client for TLS client connections
Jan 06 11:58:47 test_tls INFO    Checking client for unsupported TLS client connections
Jan 06 11:58:47 test_tls INFO    Running security.tls.v1_2_server
Jan 06 11:58:47 test_tls INFO    Connection to 10.10.10.14:443 was refused.
Jan 06 11:58:47 test_tls INFO    Failed to resolve public certificate
Jan 06 11:58:47 test_tls INFO    Connection to 10.10.10.14:443 was refused.
Jan 06 11:58:47 test_tls INFO    Failed to resolve public certificate
Jan 06 11:58:47 test_tls INFO    TLS server test results: (None, 'TLS 1.2 not validated: Failed to resolve public certificate\nTLS 1.3 not validated: Failed to resolve public certificate')
Jan 06 11:58:47 test_tls INFO    Running security.tls.v1_2_client
Jan 06 11:58:47 test_tls INFO    Validating client for TLS: 1.2
Jan 06 11:58:52 test_tls INFO    Checking client for TLS Protocol client connections
Jan 06 11:58:57 test_tls INFO    No client hello packets detected
Jan 06 11:58:57 test_tls INFO    Checking client for non-TLS client connections
Jan 06 11:59:02 test_tls INFO    Checking client for TLS client connections
Jan 06 11:59:07 test_tls INFO    Checking client for unsupported TLS client connections
Jan 06 11:59:17 test_tls INFO    Running security.tls.v1_3_server
Jan 06 11:59:17 test_tls INFO    Connection to 10.10.10.14:443 was refused.
Jan 06 11:59:17 test_tls INFO    Failed to resolve public certificate
Jan 06 11:59:17 test_tls INFO    Running security.tls.v1_3_client
Jan 06 11:59:17 test_tls INFO    Validating client for TLS: 1.3
Jan 06 11:59:22 test_tls INFO    Checking client for TLS Protocol client connections
Jan 06 11:59:27 test_tls INFO    No client hello packets detected
Jan 06 11:59:27 test_tls INFO    Checking client for non-TLS client connections
Jan 06 11:59:32 test_tls INFO    Checking client for TLS client connections
Jan 06 11:59:37 test_tls INFO    Checking client for unsupported TLS client connections
Jan 06 11:59:47 test_tls INFO    Writing results to /runtime/output/tls-result.json
Jan 06 11:59:57 test_tls INFO    Module report generated at: /runtime/output/tls_report.html
Jan 06 12:00:01 test_orc DEBUG   Test module did not produce a markdown module report
Jan 06 12:00:01 test_orc DEBUG   Adding module report for module tls
Jan 06 12:00:01 test_orc INFO    Test module tls has finished

In the not working 2.1.1 against this DUT, the TLS module times out ~9 minutes after module start.

Mar 07 14:54:24 test_module INFO    Starting TLS Module
Mar 07 14:54:27 test_tls INFO    Resolved device IP: 10.10.10.15
Mar 07 14:54:27 test_tls INFO    Running security.tls.v1_0_client
Mar 07 14:54:27 test_tls INFO    Validating client for TLS: 1.0
Mar 07 14:54:37 test_tls INFO    Checking client for TLS Protocol client connections
Mar 07 14:54:42 test_tls INFO    No client hello packets detected
Mar 07 14:54:42 test_tls INFO    Checking client for non-TLS client connections
Mar 07 14:54:48 test_tls INFO    Checking client for TLS client connections
Mar 07 14:54:53 test_tls INFO    Checking client for unsupported TLS client connections
Mar 07 14:54:53 test_tls INFO    Validating client for TLS: 1.1
Mar 07 14:54:58 test_tls INFO    Checking client for TLS Protocol client connections
Mar 07 14:55:03 test_tls INFO    No client hello packets detected
Mar 07 14:55:03 test_tls INFO    Checking client for non-TLS client connections
Mar 07 14:55:08 test_tls INFO    Checking client for TLS client connections
Mar 07 14:55:13 test_tls INFO    Checking client for unsupported TLS client connections
Mar 07 14:55:13 test_tls INFO    Validating client for TLS: 1.2
Mar 07 14:55:20 test_tls INFO    Checking client for TLS Protocol client connections
Mar 07 14:55:26 test_tls INFO    No client hello packets detected
Mar 07 14:55:26 test_tls INFO    Checking client for non-TLS client connections
Mar 07 14:55:31 test_tls INFO    Checking client for TLS client connections
Mar 07 14:55:36 test_tls INFO    Checking client for unsupported TLS client connections
Mar 07 14:55:36 test_tls INFO    Validating client for TLS: 1.3
Mar 07 14:55:42 test_tls INFO    Checking client for TLS Protocol client connections
Mar 07 14:55:47 test_tls INFO    No client hello packets detected
Mar 07 14:55:47 test_tls INFO    Checking client for non-TLS client connections
Mar 07 14:55:52 test_tls INFO    Checking client for TLS client connections
Mar 07 14:55:57 test_tls INFO    Checking client for unsupported TLS client connections
Mar 07 14:55:57 test_tls INFO    Running security.tls.v1_2_server
Mar 07 14:55:57 test_tls INFO    Scanning for HTTP ports on 10.10.10.15
Mar 07 14:56:35 test_tls INFO    Running security.tls.v1_2_client
Mar 07 14:56:35 test_tls INFO    Validating client for TLS: 1.2
Mar 07 14:57:22 test_tls INFO    Checking client for TLS Protocol client connections
Mar 07 14:58:08 test_tls INFO    No client hello packets detected
Mar 07 14:58:08 test_tls INFO    Checking client for non-TLS client connections
Mar 07 14:59:09 test_tls INFO    Checking client for TLS client connections
Mar 07 14:59:55 test_tls INFO    Checking client for unsupported TLS client connections
Mar 07 15:00:53 api      DEBUG   Received reports list request
Mar 07 15:01:30 test_tls INFO    Running security.tls.v1_3_server
Mar 07 15:01:30 test_tls INFO    Scanning for HTTP ports on 10.10.10.15
Mar 07 15:02:08 test_tls INFO    Running security.tls.v1_3_client
Mar 07 15:02:08 test_tls INFO    Validating client for TLS: 1.3
Mar 07 15:03:09 test_orc ERROR   Module timeout exceeded, killing module: tls
Mar 07 15:03:09 tls_module DEBUG   Stopping module tr-ct-tls-test
Mar 07 15:03:09 tls_module DEBUG   Killing container: tr-ct-tls-test
Mar 07 15:03:11 tls_module DEBUG   Container stopped: tr-ct-tls-test
Mar 07 15:03:11 test_orc ERROR   Error occurred whilst obtaining results for module tls
Mar 07 15:03:11 test_orc ERROR   Expecting value: line 1 column 1 (char 0)
Mar 07 15:03:11 test_orc DEBUG   Test module did not produce a markdown module report
Mar 07 15:03:11 test_orc DEBUG   Test module did not produce a html module report
Mar 07 15:03:11 test_orc DEBUG   Test module did not produce a module template

Does the default TLS module timeout need extending to cater for the change in TLS testing in 2.1.1 and/or is the ~9 minutes longer than expected for a device that does not use TLS?

[duncangreene]

I increased TLS module timeout to 20 minutes and ran against a device that does support TLS on port 443.

The TLS module took ~18 minutes from start to finish.

Mar 12 19:43:01 tls_module DEBUG   Starting module TLS
Mar 12 19:43:01 tls_module DEBUG   Network: bridge, image name: testrun/tls-test,
                       container name: tr-ct-tls-test
Mar 12 19:43:03 test_orc DEBUG   Attaching test module to the network
Mar 12 19:43:03 net_orc  DEBUG   Attaching test module  TLS to device bridge
Mar 12 19:43:03 ovs_ctrl DEBUG   Adding port tr-d-t-tls to OVS bridge: tr-d
Mar 12 19:43:19 test_module INFO    Starting TLS Module
Mar 12 19:43:21 test_tls INFO    Resolved device IP: 10.10.10.13
Mar 12 19:43:21 test_tls INFO    Running security.tls.v1_0_client
Mar 12 19:43:21 test_tls INFO    Validating client for TLS: 1.0
Mar 12 19:43:27 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 19:43:32 test_tls INFO    No client hello packets detected
Mar 12 19:43:32 test_tls INFO    Checking client for non-TLS client connections
Mar 12 19:43:37 test_tls INFO    Checking client for TLS client connections
Mar 12 19:43:42 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 19:43:42 test_tls INFO    Validating client for TLS: 1.1
Mar 12 19:43:47 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 19:43:52 test_tls INFO    No client hello packets detected
Mar 12 19:43:52 test_tls INFO    Checking client for non-TLS client connections
Mar 12 19:43:58 test_tls INFO    Checking client for TLS client connections
Mar 12 19:44:03 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 19:44:03 test_tls INFO    Validating client for TLS: 1.2
Mar 12 19:44:08 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 19:44:13 test_tls INFO    No client hello packets detected
Mar 12 19:44:13 test_tls INFO    Checking client for non-TLS client connections
Mar 12 19:44:18 test_tls INFO    Checking client for TLS client connections
Mar 12 19:44:23 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 19:44:23 test_tls INFO    Validating client for TLS: 1.3
Mar 12 19:44:28 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 19:44:34 test_tls INFO    No client hello packets detected
Mar 12 19:44:34 test_tls INFO    Checking client for non-TLS client connections
Mar 12 19:44:39 test_tls INFO    Checking client for TLS client connections
Mar 12 19:44:44 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 19:44:44 test_tls INFO    Running security.tls.v1_2_server
Mar 12 19:44:44 test_tls INFO    Scanning for HTTP ports on 10.10.10.13
Mar 12 19:46:17 test_tls INFO    Checking HTTP ports on 10.10.10.13: [443]
Mar 12 19:46:17 test_tls INFO    Port 443: HTTPS
Mar 12 19:46:17 test_tls INFO    Inspecting Service on port 443: HTTPS
Mar 12 19:46:18 test_tls INFO    Device certificate:
Mar 12 19:46:18 test_tls INFO    Certificate valid from: 2024-08-30 15:44:32 To 2025-08-30 15:44:32
Mar 12 19:46:18 test_tls INFO    Key Length: 2048
Mar 12 19:46:18 test_tls INFO    Key Type: RSA
Mar 12 19:46:18 test_tls INFO    Checking for valid signature from authorized Certificate Authorities
Mar 12 19:46:18 test_tls INFO    SSL error occurred: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)
Mar 12 19:46:18 test_tls INFO    Authorized Certificate Authority signature not present
Mar 12 19:46:18 test_tls INFO    Checking for authorized CA certificate chain
Mar 12 19:46:18 test_tls INFO    Validating certificate chain
Mar 12 19:46:18 test_tls INFO    Resolving CA Issuer
Mar 12 19:46:18 test_tls INFO    CA Issuers resolved: None
Mar 12 19:46:18 test_tls INFO    Authorized Certificate Authority signature not present
Mar 12 19:46:18 test_tls INFO    Root Certs Found: 0
Mar 12 19:46:18 test_tls INFO    Certificate validated: False
Mar 12 19:46:18 test_tls INFO    Device certificate:
Mar 12 19:46:18 test_tls INFO    Certificate valid from: 2024-08-30 15:44:32 To 2025-08-30 15:44:32
Mar 12 19:46:18 test_tls INFO    Key Length: 2048
Mar 12 19:46:18 test_tls INFO    Key Type: RSA
Mar 12 19:46:18 test_tls INFO    Checking for valid signature from authorized Certificate Authorities
Mar 12 19:46:18 test_tls INFO    SSL error occurred: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)
Mar 12 19:46:18 test_tls INFO    Authorized Certificate Authority signature not present
Mar 12 19:46:18 test_tls INFO    Checking for authorized CA certificate chain
Mar 12 19:46:18 test_tls INFO    Validating certificate chain
Mar 12 19:46:18 test_tls INFO    Resolving CA Issuer
Mar 12 19:46:18 test_tls INFO    CA Issuers resolved: None
Mar 12 19:46:18 test_tls INFO    Authorized Certificate Authority signature not present
Mar 12 19:46:18 test_tls INFO    Root Certs Found: 0
Mar 12 19:46:18 test_tls INFO    Certificate validated: False
Mar 12 19:46:18 test_tls INFO    TLS server test results: (False, 'TLS 1.2 not validated on port 443: Certificate has a valid time range\nRSA key length passed: 2048 >= 2048\nDevice certificate has not been signed\nTLS 1.3 not validated on port 443: Certificate has a valid time range\nRSA key length passed: 2048 >= 2048\nDevice certificate has not been signed')
Mar 12 19:46:18 test_tls INFO    Running security.tls.v1_2_client
Mar 12 19:46:18 test_tls INFO    Validating client for TLS: 1.2
Mar 12 19:47:01 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 19:47:45 test_tls INFO    No client hello packets detected
Mar 12 19:47:45 test_tls INFO    Checking client for non-TLS client connections
Mar 12 19:48:29 test_tls INFO    Checking client for TLS client connections
Mar 12 19:49:12 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 19:50:39 test_tls INFO    Running security.tls.v1_3_server
Mar 12 19:50:39 test_tls INFO    Inspecting Service on port 443: HTTPS
Mar 12 19:50:39 test_tls INFO    Device certificate:
Mar 12 19:50:39 test_tls INFO    Certificate valid from: 2024-08-30 15:44:32 To 2025-08-30 15:44:32
Mar 12 19:50:39 test_tls INFO    Key Length: 2048
Mar 12 19:50:39 test_tls INFO    Key Type: RSA
Mar 12 19:50:39 test_tls INFO    Checking for valid signature from authorized Certificate Authorities
Mar 12 19:50:39 test_tls INFO    SSL error occurred: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)
Mar 12 19:50:39 test_tls INFO    Authorized Certificate Authority signature not present
Mar 12 19:50:39 test_tls INFO    Checking for authorized CA certificate chain
Mar 12 19:50:39 test_tls INFO    Validating certificate chain
Mar 12 19:50:39 test_tls INFO    Resolving CA Issuer
Mar 12 19:50:39 test_tls INFO    CA Issuers resolved: None
Mar 12 19:50:39 test_tls INFO    Authorized Certificate Authority signature not present
Mar 12 19:50:39 test_tls INFO    Root Certs Found: 0
Mar 12 19:50:39 test_tls INFO    Certificate validated: False
Mar 12 19:50:39 test_tls INFO    Running security.tls.v1_3_client
Mar 12 19:50:39 test_tls INFO    Validating client for TLS: 1.3
Mar 12 19:51:22 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 19:52:05 test_tls INFO    No client hello packets detected
Mar 12 19:52:05 test_tls INFO    Checking client for non-TLS client connections
Mar 12 19:52:50 test_tls INFO    Checking client for TLS client connections
Mar 12 19:53:33 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 19:54:59 test_tls INFO    Writing results to /runtime/output/tls-result.json
Mar 12 20:00:49 test_tls INFO    Module report generated at: /runtime/output/tls_report.j2.html
Mar 12 20:00:58 test_orc DEBUG   Test module did not produce a markdown module report
Mar 12 20:00:58 test_orc DEBUG   Test module did not produce a html module report
Mar 12 20:00:58 test_orc DEBUG   Adding module template for module tls

[duncangreene]

I increased TLS module timeout to 20 minutes and ran against the DUT in the OP (i.e. the one that doesn't support TLS at all).

The TLS module took ~19 minutes 30 seconds from start to finish, for a device that does not support TLS. In 2.1 this took ~3 minutes.

Mar 12 20:37:09 tls_module DEBUG   Starting module TLS
Mar 12 20:37:09 tls_module DEBUG   Network: bridge, image name: testrun/tls-test,
                       container name: tr-ct-tls-test
Mar 12 20:37:11 test_orc DEBUG   Attaching test module to the network
Mar 12 20:37:11 net_orc  DEBUG   Attaching test module  TLS to device bridge
Mar 12 20:37:11 ovs_ctrl DEBUG   Adding port tr-d-t-tls to OVS bridge: tr-d
Mar 12 20:37:26 test_module INFO    Starting TLS Module
Mar 12 20:37:28 test_tls INFO    Resolved device IP: 10.10.10.15
Mar 12 20:37:28 test_tls INFO    Running security.tls.v1_0_client
Mar 12 20:37:28 test_tls INFO    Validating client for TLS: 1.0
Mar 12 20:37:34 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 20:37:39 test_tls INFO    No client hello packets detected
Mar 12 20:37:39 test_tls INFO    Checking client for non-TLS client connections
Mar 12 20:37:44 test_tls INFO    Checking client for TLS client connections
Mar 12 20:37:49 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 20:37:49 test_tls INFO    Validating client for TLS: 1.1
Mar 12 20:37:54 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 20:38:00 test_tls INFO    No client hello packets detected
Mar 12 20:38:00 test_tls INFO    Checking client for non-TLS client connections
Mar 12 20:38:05 test_tls INFO    Checking client for TLS client connections
Mar 12 20:38:10 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 20:38:10 test_tls INFO    Validating client for TLS: 1.2
Mar 12 20:38:15 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 20:38:20 test_tls INFO    No client hello packets detected
Mar 12 20:38:20 test_tls INFO    Checking client for non-TLS client connections
Mar 12 20:38:26 test_tls INFO    Checking client for TLS client connections
Mar 12 20:38:31 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 20:38:31 test_tls INFO    Validating client for TLS: 1.3
Mar 12 20:38:36 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 20:38:41 test_tls INFO    No client hello packets detected
Mar 12 20:38:41 test_tls INFO    Checking client for non-TLS client connections
Mar 12 20:38:46 test_tls INFO    Checking client for TLS client connections
Mar 12 20:38:52 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 20:38:52 test_tls INFO    Running security.tls.v1_2_server
Mar 12 20:38:52 test_tls INFO    Scanning for HTTP ports on 10.10.10.15
Mar 12 20:39:31 test_tls INFO    Running security.tls.v1_2_client
Mar 12 20:39:31 test_tls INFO    Validating client for TLS: 1.2
Mar 12 20:40:18 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 20:41:06 test_tls INFO    No client hello packets detected
Mar 12 20:41:06 test_tls INFO    Checking client for non-TLS client connections
Mar 12 20:42:05 test_tls INFO    Checking client for TLS client connections
Mar 12 20:42:52 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 20:44:28 test_tls INFO    Running security.tls.v1_3_server
Mar 12 20:44:28 test_tls INFO    Scanning for HTTP ports on 10.10.10.15
Mar 12 20:45:12 test_tls INFO    Running security.tls.v1_3_client
Mar 12 20:45:12 test_tls INFO    Validating client for TLS: 1.3
Mar 12 20:46:45 test_tls INFO    Checking client for TLS Protocol client connections
Mar 12 20:48:20 test_tls INFO    No client hello packets detected
Mar 12 20:48:20 test_tls INFO    Checking client for non-TLS client connections
Mar 12 20:50:21 test_tls INFO    Checking client for TLS client connections
Mar 12 20:51:55 test_tls INFO    Checking client for unsupported TLS client connections
Mar 12 20:55:04 test_tls INFO    Writing results to /runtime/output/tls-result.json
Mar 12 20:56:40 test_tls INFO    Module report generated at: /runtime/output/tls_report.j2.html
Mar 12 20:56:42 test_orc DEBUG   Test module did not produce a markdown module report
Mar 12 20:56:42 test_orc DEBUG   Test module did not produce a html module report
Mar 12 20:56:42 test_orc DEBUG   Adding module template for module tls

[jboddey]

We added a full tcp port scan to the TLS module to find http servers on ports other than 80 and 443, I believe this would be the reason, we probably just need to increase the timeout further.

[jboddey]

Closing this issue, please increase the module timeout in the system.json file.