TLS certificate validation failed: error 75788

[cabricharme]

In multi-site all-Meraki environment with a site-to-site VPN, we're getting "TLS certificate validation failed: error 75788" errors in a specific application, when Meraki firewall is enabled, e.g.:

:STD: 2024-03-12T12:07:18.76048 ERROR 0/<Aloha01>:4 RdfServerClient [WW] Command result 'e95*****-1**c-4**1-8**b-f*********6' failed to send
nsoftware.SecureBlackbox.SecureBlackboxHttpclientException: Failed to post data
[SBHTTPSClient.EElHTTPSConnectionShutdownError] TLS certificate validation failed: error 75788; the connection will be terminated (100353/0x18801) ---> SBHTTPSClient.EElHTTPSConnectionShutdownError: TLS certificate validation failed: error 75788; the connection will be terminated

This seems to affect just one application out of a gazillion we use. Any idea what could be causing this?

Notes:

• The errors are isolated to a specific application, NCR Command Center. The application is effectively a C&C (Command & Control) center akin to MS Server Manager, it "talks" to "clients" in remote sites and asks them to do things like display the contents of a file system, execute a certain command, send a message, restart the client, etc.
  • (tried asking on the application support forum - didn't get too far)
• All other applications and services do not seem to be affected, and it's a whole lot of them: active directory, RDS, vSphere/vCenter/ESXis, file transfers, etc.
• When Meraki firewall is temporarily dropped, the errors and the issues go away.
• See nothing in Meraki logs, and nothing in its firewall or other configuration that could be causing this.
• The issue started around February 2024, and so far we could not quite correlate it with any configuration changes or other events.
• The application appears to be using N Software Secure Black Box application libraries for those TLS connections. N Software does have a KB article describing the issue (thanks @alemabrahao!) - yet it doesn't explain why would cert validation work w/o Meraki, and stop working with it.
• I am not sure which TLS certificate is failing validation by the application. Is there a good way to figure this out? Is it Meraki's, or the the application's?
• Is it that Meraki firewall is dropping (very) specific TLS packets that result in failed validation? Anything to do with TLS version and Meraki not supporting some of the older, vulnerable ones?
  • (There is this: "FAQ: Meraki Authentication to Require TLS 1.2 or Later Version" - yet it affects wireless devices only, it seems, and the devices in question are all on a wired network.)
• I am not the network admin - more of a generalist - but do have read-only access to our Meraki systems.

Thanks for any ideas!

P.S. If no specific root cause comes to mind, would love for you to try and assist me in isolation and troubleshooting:

• Is there an easy way to see which certificate fails to get validated?
• Is there an easy way to see which TLS version is being used by the application?

[alemabrahao]

The error message “TLS certificate validation failed: error 75788” indicates that the TLS certificate used by the application failed to validate. This could be due to a variety of reasons, such as an expired certificate, a self-signed certificate not trusted by the system, or a certificate not matching the server’s domain name.

https://cdn.nsoftware.com/help/SBF/cs/pg_tlscertvalidation.htm

About FAQ: Meraki Authentication to Require TLS 1.2 or Later Version  this change primarily affects wireless devices using 802.1x for Meraki Authentication. If your application is running on a wired network, this change might not be the direct cause of your issue, but it’s worth checking the TLS version used by your application.

[cabricharme]

Thank you! I did see that document while googling around - and don't yet see how it could be helpful in figuring out what part of Meraki (or any other configuration) results in certificate validation failures - and only when Meraki firewall is enabled. (I am not even sure how it works: why would Meraki get itself in the middle of a cert validation process?)

[cabricharme]

Packet capture (during an induced failure) shows a TLSv1.2 "Certificate Unknown" error to a specific endpoint, however testing that endpoint's certificate in the browser or via PowerShell on the affected server (where the validation appears to be failing) shows no red flags - the endpoint is reachable, the certificate valid.

    TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Certificate Unknown)
        Content Type: Alert (21)
        Version: TLS 1.2 (0x0303)
        Length: 2
        Alert Message
            Level: Fatal (2)
            Description: Certificate Unknown (46)

We'll work with Meraki and application vendor support to keep troubleshooting the issue.

[cabricharme · Accepted solution]

Adding IP address ranges for CRLs (Certificate Revocation Lists) to Layer 3 firewall "allow" rules fixed the issue.

The CRL FQDNs and IP ranges were provided by the application vendor.

The mystery: these FQDNs and IPs did not show up in the packet capture during the failure - unless I did not capture the traffic or search the packet capture correctly.