You can use the stunnel utility to verify a specific key.
You can extract the public key from a TLS session using a variety of means, and record it to a file. It will look like this:
Then you configure stunnel with something like this:
echo 'FIPS = no
client = yes
verify = 4
cafile = /path/to/publickey.pem
accept = 127.0.0.1:65432
connect = your.remote.server.com:443' > stunnel-verify.conf
Then run stunnel with that configuration, and it will refuse to connect to any server that does not have the correct private key.
Your local client will need to make a plaintext connection to port 65432 on localhost. If the connection is successful and the remote server responds, you can be sure that the remote keypair has been verified.