If you have a Service that you are accessing from outside (anywhere in the World), it’s probably a good idea to access it through a secure connection.

If you followed one of the previous articles: Access a Service from Anywhere with DuckDNS

you are in a situation where you can access your Service through a DNS provides (such as DuckDNS), but using an non-secure HTTP connection.

In this article I’ll explain the concept behind SSL/HTTPS and how to achieve it.

SSL and HTTPS

SSL to encrypt our connection (from Wikipedia):

Cryptographic protocol designed to provide communications security over a computer network.

In conjunction with it you’ll hear the other acronym HTTPS that is instead simply identifying that the protocol used to reach the page/website is secured with an SSL Certificate.

I will look into two ways to achieve this:

• One is the simplest and quickest one that does the minimum to reach the goal and will be very useful to understand the key concepts that comes into play here.
• The second one is built on top of the previous and expands it making it more professional in the way you’ll see.

Splitting this into this two section will help in understanding clearly what’s going on and why we need to do what we’ll do, without putting too much concept all together confusing the whole thing.

The key difference between the two is the creation of the Certificates used to encrypt the connection.

In the first case we’ll just create them ourselves, while in the second we’ll use a Certificate Authority.

Again, the difference is the validity of the certificate and how much we (or anyone for what it matters) can trust it… and therefore how much is the connection trustworthy.

One is the proper fully fledged way that uses certificates requested through a real Certificate Authority. For this the common solution is to use the free https://letsencrypt.org/: “Let’s Encrypt is a free, automated, and open Certificate Authority.”

The other way (a little bit quicker) is to self-sign our certificates and use them.

The drawback of the second solution is only that the authority (us) cannot be verified and therefore the browser will raise a warning of the danger… but we know it’s us ;)

I’ll use the second quick option for now, so that the main concept will be clear (not too much magic) and is both good enough for me and also allows to do the next step if we want to, focusing only on the certification step.

Create the (Self-Signed) SSL Certificates

So, let’s start with the first version and let’s create our self-signed certificate!

To create the certificate we’ll use “openssl“. We’ll specify few parameters with the following meaning:

• “req“: From the manual: “The req command primarily creates and processes certificate requests in PKCS#10 format. It can additionally create self signed certificates for use as root CAs for example.“
• “-days 365“: I will create a certificate that will expire in 365 days. By default this would be 30 days
• “-sha256“: That’s just the secure hash algorithm
• “-newkey“: generate a new private key
• “rsa:4096“: is the type and length of the key
• “-nodes“: it stands for “no des” and it’s used to generate the private key without encrypting it (with a password). Despite the security concerns is commonly used as option, mainly because you would need to type the password to start services and therefore would create problem with auto-restarting services
• “-keyout privkey.pem“: The generated file that will contain the private key will be: “privkey.pem“
• “-x509 -out certificate.pem“: The certificate will be stored in the file: “certificate.pem“

That’s the complete command:

When you execute it, the system will prompt you with some questions about the certificate.

In this case you can answer whatever you want (check the “Country Name”, “Common Name”, etc. in the next block to see what I put there as an example).

That’s the final output you should expect from the execution of the command:

As I mentioned above the certificate expiration date is 30 days by default but we specified “-days 356” to extend the validity to 1 year… so let’s check it out (for fun and to learn!)

You can check the expiration date with:

That is in fact 1 year from now! (the time I’m working on this article).

Anyway, job done!

We have the certificate and the associated private key:

Configure the service

At this point it’s generally a question of configuring the specific service, in order to let it know where are these files and allow it to use them to provide the service over https connections.

The way of achieve this varies from service to service and therefore you’ll need to check the specific documentation provided by your service.

Test it out

At this point, if you reach your service with https you will immediately notice a big warning:

To confirm which certificate the browser is complaining about, and to see how everything makes sense and goes back together, click on the “Show Details” button:

and then on “view the certificate“:

as you can see, that’s exactly the certificate we signed with all the data we put above… pretty cool uh?

Hopefully this clarifies a lot what’s going on there and the reason of everything we did above.

You can instead click on “visit this website” to go ahead and reach the page we are looking for.

Similarly, as another example, if you use Chrome you’ll see something like this:

Once again, click on “Advanced“:

and then on “Proceed to …” to access your Home Assistant dashboard with a secure connection.

Again, on the address bar Chrome will make sure you remember about the non-trusted certificate…

but hopefully you know you can thrust yourself, right?

So in theory you could just be happy and safely access your Service in this way.

If you are ready instead to work more on it, because you don’t like the scary warnings, the need to press multiple buttons to access your website or you need to make it the website accessible to other people and you don’t want to scare them out, then keep reading.

Make a Trustworthy Certificate

As mentioned at the start, there are Certification Authorities that allow to create trusted certificates.

Using them will avoid the warnings that we currently receive.

The Certificate Authority I’ll use for this is https://letsencrypt.org/:

Let’s Encrypt is a free, automated, and open Certificate Authority.

The process of creating a certificate and updating it (when it expires) is a little bit involved and while we could do it manually, for this step I’ll use a script that takes care of a lot of details directly for us.

You can find it there: https://dehydrated.io/

Dehydrated is a client for signing certificates with an ACME-server (currently only provided by Let’s Encrypt) implemented as a relatively simple bash-script.

The bash-script it’s referring to is here: https://github.com/lukas2511/dehydrated

Dehydrated

If you want to grab the code from github you could checkout it with git or svn at the url: “https://github.com/lukas2511/dehydrated.git”

Just grab the code

For simplicity (especially if you don’t have git) I can just grab dehydrated downloading the files (we don’t have intention to modify/check-in or update anything and we can  grab it again in the future if needed).

Anyway, I will put it inside a subfolder “dehydrated” placed inside the same “certs” folder I created before (too keep all this together):

Let’s now download it straight from the source:

With git instead

With git you can simply grab the code (and it will create the dehydrated folder for you).

Go into the “certs” folder and clone the repository with:

That’s the output:

Enter in the folder to continue:

Make it executable

One important step is now to make the script executable.

This can be easily done with the command:

Ok, if we follow the documentation it says that before executing it we need to specify the domain we want to secure inside a file named “domains.txt” (you can read this here: “https://github.com/lukas2511/dehydrated/blob/master/docs/domains_txt.md”)

Let’s do this then:

Inside this file just write the name of your website, that should be something like this, naturally with your specific domain specified:

Once this is done we need to edit another file “config”

You can read about it here: “https://github.com/lukas2511/dehydrated#config”

Write this inside it:

Under the hood

If you are not interested into the details of what we just wrote, skip this section, otherwise bare with me.

Going back to Let’s Encrypt, let’s read more of its definition:

“To enable HTTPS on your website, you need to get a certificate (a type of file) from a Certificate Authority (CA). Let’s Encrypt is a CA. In order to get a certificate for your website’s domain from Let’s Encrypt, you have to demonstrate control over the domain. With Let’s Encrypt, you do this using software that uses the ACME protocol, which typically runs on your web host.”

Dehydrated is one of this software (well, script in this case) that use the ACME protocol we need. In fact, looking at dehydrated once again:

“Dehydrated is a client for signing certificates with an ACME-server (e.g. Let’s Encrypt) implemented as a relatively simple (zsh-compatible) bash-script. This client supports both ACME v1 and the new ACME v2 including support for wildcard certificates!”

Now, Let’s Encrypt need a demonstration that you control the domains you want to secure. For this we use the “dns-01” verification (you can read about other types of verification on their website). The details are here: https://docs.certifytheweb.com/docs/dns-validation.html

Dehydrated supports “dns-01” (https://github.com/lukas2511/dehydrated/blob/master/docs/dns-verification.md).

As described in that documentation, this type of verification requires you to be able to create a specific TXT DNS record (from wikipedia: “a type of resource record in the Domain Name System (DNS) used to provide the ability to associate arbitrary text with a host or other name, such as human readable information about a server, network, data center, or other accounting information.“) for each hostname included in the certificate.

Dehydrated then specified that we need a hook script that deploys the challenge to our DNS server.

In order to do this, dehydrated will invoke our hook script… more on this in a bit.

As a matter of fact, DuckDNS allows us to generate this TXT record.

Looking at the documentation (https://www.duckdns.org/spec.jsp):

“The TXT update URL can be requested on HTTPS or HTTP.”

and again:

“You can update your domain(s) TXT record with a single HTTPS get to DuckDNS your TXT record will apply to all sub-subdomains under your domain e.g. xxx.yyy.duckdns.org shares the same TXT record as yyy.duckdns.org”

As an example it’s given:

as you will see, we’ll do this in our dehydrated hook, using something like:

The “$4” parameter is explained in dehydrated documentation (all the parameters):

• $1 an operation name (clean_challenge, deploy_challenge, deploy_cert, invalid_challenge or request_failure) and some operands for that.

For deploy_challenge

• $2 is the domain name for which the certificate is required,
• $3 is a “challenge token” (which is not needed for dns-01), and
• $4 is a token which needs to be inserted in a TXT record for the domain.

The DuckDNS hook is the one indicated online always in the documentation: https://github.com/lukas2511/dehydrated/wiki/DNS-01-hook-for-DuckDNS

As another example, to clear the token the real important part is only the last parameter to pass to DuckDNS: “clear=true“.

Note that the challenge token is what we’ll receive from Let’s Encrypt.

So, the full logic flow is essentially the following:

• We request Let’s Encrypt (our Certificate Authority) to generate a certificate for our domain
• We receive the certificate and the challenge token to use to prove that we control the domain
• We tell DuckDNS to update the TXT record for our domain to match the token used for the challenge
• We then tell Let’s Encrypt validate the domain (more here: https://letsencrypt.org/how-it-works/)

This should help in clarifying what is this “dns-01”, how we tell DuckDNS to verify it and the meaning of the hook script.

Ok, Where were we…

If you read the section above you should know why we need the hook script. In any case, we need a hook to help with the DNS challenge.

In the same script we’ll also provide a way of restarting the Home Assistant service and this is used when the certificate is created/updated.

Create a “hook.sh” file:

The content comes from the one provided by Dehydrated as an example: https://github.com/lukas2511/dehydrated/wiki/DNS-01-hook-for-DuckDNS

I modified the “deploy_cert” section in order to restart the service using the “pi” account (because we need the sudo privileges).

Anyway, this is the content:

NOTE: change:

• <your_domain> with your specific DuckDNS domain, in the same way you did above
• <your_token> with your DuckDNS token

(if you t follow the other posts or you have a different installation, ensure to change the “deploy_cert” section as well to correctly restart the Home Assistant service)

Last thing to do is make it executable:

Checkpoint

At the end of all of this verify that this is what you have in your folder:

Register with letsencrypt

From the (dehydrated) documentation:

“Before any certificates can be requested, Dehydrated needs to acquire an account with the Certificate Authorities.

Optionally, an email address can be provided. It will be used to e.g. notify about expiring certificates. You will usually need to accept the Terms of Service of the CA.

Dehydrated will notify if no account is configured.

Run with –register –accept-terms to create a new account.”

The license agreement for Let’s Encrypt (at the time of writing) is there: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf

If you run dehydrated without the “–accept-terms” you can see the latest terms link.

If you want to register a mail, edit your config file adding (with your email):

Time to register!

When you are ready run dehydrated with the registration options:

That should produce this output:

It created a new folder “accounts” that will contain a folder with the information about the registrations with letsencrypt.

Create the certificate

We can now happily created the wanted certificate!

Just run dehydrated with this option:

and this should be the output:

Note that at the end the script is requesting for the sudo password for pi.

This is because the hook is trying to execute “deploy_cert” that tries to restart the service.

If you didn’t change the default hook script in the way I described above, you should receive the request of the sudo password for “myuser”. At that point you will probably need to kill the script [CTRL+C].

Everything is still ok, but you will manually need to restart the service.

Configure the service

As already mentioned in the previous section, the configuration really varies from service to service, so you’ll need to verify how to specify the certificate and key in the custom settings of your service.

The certificate and key can be found in:

Test it out

you can now open the browser and try again to reach your Service website using HTTPS:

This time the warning should have disappeared and everything should be nice and clean!

If you check the certificate this time is indeed from “Let’s Encrypt Authority” and considered valid (trustworthy):

Autorenew (cronjob)

If you check the expiration date of the certificate generated by Let’s Encrypt you’ll see that is set to 90 days.

This means that before that time we’ll have to run the dehydrated script again to refresh the certificate.

You can clearly do it manually, but is probably a good idea to automate this step.

We can easily do this setting up a cron job to execute every so often.

In this job we’ll execute the certificate creation script and this already takes care of checking the validity and:

• Do nothing: if the certificate is still valid (and more than 30 days from expiration by default)

or:

• Update the certificate otherwise

Check the hook.sh

If required, we need to check our “hook.sh” to use exactly this command where it’s trying to restart the service.

We can test that it works simply running:

Add the cron job!

I will add an automatic job that will run at 8:00 of the 1st day of every month.

The command it will execute will be the “./dehydrated -c” command that will take care of renewing the certificate if the expiration date is less than 30 days ahead.

You can create a cron job with:

Quick verification…

Note, to quickly test that the service restart will work you can put this in the crontab:

then wait for the CRON log that will notify the fact that the command has been executed (and therefore the service restarted):

that after a couple of minutes will be something like:

Anyway, as soon as you see the command executed you can kill the tail command and check the status of the service and notice that the start time should be just few seconds before.

NOTE: Remember to remove the automated job now, to avoid to constantly restart the service every minute!

Back to our real cron job

Ok, now that we know that everything work as expected, put this in the cron file instead:

then save and exit.

Job Done!

From now on, this will take care from now on of renewing the certificates when required… and restart the service when this happens! Fantastic!

Conclusion

If you followed the whole article you should now have everything clear about how connect securely to your Service and understand everything required for this.

We saw how to generate and have full control of our certificate and and also how to have any future certification done automatically for us from now on.

If you enjoyed this article, found it interesting or just learned something new (or simply liked it) please share it, comment or feel free to if you think I deserved it or you simply want to show your appreciation :). In any case thanks for passing by and until next time!