Curl is the “go to” tool for many application delivery engineers. The ability to easily send test payloads and see HTTP response headers quickly and easily makes this tool indispensable when troubleshooting or learning application delivery controllers.
When testing something like host headers on apache web servers, one often will include an HTTP host header with something like:
curl -H "Content-Type: application/json" -H "X-Auth: abc1234567890" -H "Host: www.foo.com" https://192.168.1.111
While the host header has long been an important piece of the equation, it is not sufficient for testing SNI enabled servers.
Server Name Indication (SNI), as you probably know, allows us to stretch the IPv4 space even further by allowing a site with a single IP address to serve up a number distinct and separate SSL certificates. In days of old, one would need a distinct IP address for each SSL certificated site or use Subject Alternate Names with a single IP address. From a hosting providers point of view, the latter option was unappealing as a provider might host two competitors and they would be served from the same SSL certificate. So while SAN does have some good use cases, it is often unsuitable depending on the business case involved and some other drawbacks that I will not get into here.
So SNI is rapidly becoming the defacto standard for implementing multiple SSL certificated sites using a minimal number of IPv4 addresses. We need a way to test them.
Case in point. I was troubleshooting health checks failing from a Microsoft ADFS front end system recently.
curl --header 'Host: adfs.mysite.com' https://192.168.1.100
The behavior from the server was to fail on the SSL handshake. Hmmmm. Strange. However, on the Windows device, when modifying the local hosts file and pointing adfs.mysite.com to 192.168.1.100 and trying this in a web browser, the site worked!
This is SNI in action. In this particular case, the ADFS site was not set up to serve a default certificate in the case that an SNI request was not presented. This meant that when a request came in with an invalid or missing SNI request, there was no default certificate to present to the client and we essentially had a web server that had no certificate bound to the service. As a result, any attempt at an SSL handshake would fail after the CLIENT HELLO.
So, we have seen that using the HOST header does not satisfy the SNI requirement. Not to fear, curl does have a solution for this. There is an option that is used if you want to test a host without the need to modify your DNS or host file.
--resolve [DOMAIN]:[PORT]:[IP]
The resolve option adds some interesting flexibility to our CURL testing.
- You don’t need to edit your hosts file to resolve and use the domain name in your curl tests
- You can use an alternate port for your HTTP request without putting the port in the URL
- You can use the FQDN rather than just an IP address in your URL (Hint: SNI)
Let’s see how this changes our testing of the ADFS server:
curl --header 'Host: adfs.mysite.com' https://192.168.1.100 curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 192.168.1.100:443
Here, we see the server basically tell us “there is no SSL certificate paired to the SNI name of ‘192.168.1.100’ “. Again, SNI is ignoring the host header, because the host header only gets sent as part of the HTTP call AFTER the SSL session has been established. Therefore, the host header is useless for SNI as during the SSL handshake, the server needs to determine WHICH ssl certificate to use to set up the session.
Instead, SNI is using the FQDN presented in the URL. This is a client behavior and is inserted into the SSL CLIENT HELLO payload as seen in the following screen shot from a Wireshark capture that I did of a curl request to my site.
curl https://www.hacksbrain.com
When making a curl request without an FQDN in the URL, the SNI extension is not sent as part of the CLIENT HELLO.
So, if the FQDN to the SNI enabled site resolves to a public IP address and one needs to test that site with curl from within the environment using the local IP address, one would need to edit the local hosts file to point that FQDN to the local IP address rather than let DNS resolve the FQDN to the public IP address. This can cause some problems with other parts of the application or a user might not have the needed permissions to edit the local host file.
Let’s see how the –resolve option changes this for us.
curl -k -I --resolve adfs.mysite.com:443:192.168.1.100 https://adfs.mysite.com/ HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=us-ascii Server: Microsoft-HTTPAPI/2.0 Date: Sun, 26 Aug 2018 21:19:24 GMT Connection: close
Ah! This is a much different response than “OpenSSL SSL_connect: SSL_ERROR_SYSCALL”. The presence of an HTTP 404 response indicates that the SSL session was properly setup and that the HTTP request was sent through the SSL tunnel and an HTTP response was received from the SNI enabled web site.
So, add this handy curl option to your toolbox
--resolve fqdn:port:ipaddress
Because if you have not yet come across an SNI enabled server that you need to troubleshoot, rest assured, you will in the future!