Certificates from Scratch - X.509 Certificates explained
What are X.509 Certificates? What is a “Certification Authority” or CA? How can we create our own CA? How can we sign our own Server certificates? How does LetsEncrypt work? How do private and public keys work? What is a certificate Chain or a Chain of Trust? The answers are in this video.
The XCA Tool can be obtained here
More Info on my Cheat Sheet Repo on Github
Watch the video on YouTube
Click to view the entire transcript
(about certificates)
We all know certificates, and we are using them on a daily basis. Even if we don’t necessarily call them certificates. Let me give you two examples. First one – if I browse to my own web site www.onemarcfifty.com then I see this little padlock icon here in the address bar. If I click on it, then my browser tells me that my connection is secure and that I have a valid certificate. Hmm. Yeah that’s great but what does it actually mean? What is secure? And what am I protected from? We’ll come to that – but first let me tell you a little story.
This is my nephew. He’s five years old and he has a kick scooter. One day I visit my brother in law and we are all sitting on the veranda. My nephew drives by and starts making car driving noises. I was up to the game, stood up and said “hold it citizen – may I see your driver’s license please?”. Now – my nephew is smart and seemed to be prepared. So he shows me this. I try to maintain a strict face and say “Thank you sir, your license does check out. Please continue and have a nice day”.
I invented this story. But it shows nicely what a certificate is. A certificate is a document that is linked to an identity. In this case “Marc’s Nephew”. So Marc’s Nephew can use this to prove his identity, but no one else can. A certificate has a subject and/or a purpose. In this case it proves that the person using it is authorized to drive a kick scooter. It could not be used to fly an airplane. It also has an issuer. In this case it’s my Nephew. It is a so called self-signed certificate. Because he issued it to himself. Anyone who trusts the issuer would in turn trust the certificate. But would this license be accepted in the real world – if he drove a real car on a real road? Presumably no. If we compare this license to a real driver’s license in the real world however, then there’s actually only one single difference here. The issuer. A driver’s license in the real world would be issued by the government typically. Or by any authority that is authorized by the government, such as a state or a county or a city and so on. Plus - those certificates have a certain number of features which makes it difficult to falsify or copy them. This example from the swiss government shows some of these features on the swiss driver's license.
(certificate chains – Certificate Autority/CA)
Back to the certificate on my web site onemarcfifty.com. How can it be that everyone on this planet trusts this certificate? Let me examine it. For this, I will use a cool software called XCA by Christian Hohnstädt. It’s awesome and it’s free open source. If we browse to www.hohnstaedt. HYPERLINK "http://www.hohnstaedt.de/"de, then we can download it for various platforms such as linux, MacOS or Windows. There’s even a portable version of this. Guys – all links to the software are in the description of the video. So let me go to my blog site, then click on the padlock and view the details of the certificate. I will then export it into a file. As a file format with Google Chrome I use PKCS#7 certificate chain. Because that format includes the most information. In Firefox you can save it as a full chain PEM file by just clicking on that link that it shows on the certificate detail page.
Next, I’ll use XCA to import it. Just before I can use XCA I need to quickly generate a new database for certificates and keys – that can be any file anywhere. And of course it needs to be protected with a password. Now I click on import, then PKCS#7 and select the file that I had just exported from my browser. Here we go. On the certificates tab I can now see the details and – equally important – I can see the chain of trust. The full certificate chain. My certificate www.onemarcfifty.com had been issued by someone or something called R3. And that certificate had been issued by a corpus called ISRG Root X1. Let’s go bottom up. I am interested in two fields here. The subject and the issuer. The subject of my certificate is www.onemarcfifty.com. The issuer is R3 who – surprise – belong to Let’s Encrypt. Let me go one level up by clicking on this green R3 in the Signature field. Here the subject of the R3 certificate is identical to the issuer of the onemarcfifty.com certificate. And the issuer of this one is of course the Internet Security Research Group or short ISRG. Again one level up to the root certificate, the so called Certificate Authority or CA. And – surprise – this CA certificate which is at the root level of the chain appears to be self signed! The issuer and the subject are identical. But wait! That one is self signed just like my nephew’s kick scooter license, right? And everyone trusts that one? How can that be? That sounds wrong.
Well, the secret is lifted if we check on the Root Certificate Authorities that my browser or my operating system trusts out of the box. On my Linux machine I can find them here in /usr/share/ca-certificates/mozilla. And there we go, there are a lot of certificates and of course there is the ISRG Root X1. So it’s part of the well known companies or organizations that are trusted by the common browser products. They are commonly trusted. And – we just learned one more thing – a CA or Certification authority is just a self signed certificate really. In your browser, you can find them in the settings under the security options. Here in Chrome you go to settings, then Privacy and security, click on security, then scroll down to Manage certificates and then select the “Authorities” tab. You can as well export them from there with the context menu. In Firefox, you can find this under Settings, Privacy and Security, then scroll all the way down to Certificates and click on “View Certificates”. Again, select the “Authorities” tab. Select a certificate and you can export it using the Export button at the bottom.
In other words – the “trust” for those self signed root CA certificates is built into the products that use them. Typically, that’s your browser.
(private keys)
A Server certificate that is signed or issued by such a trusted CA can prove to everyone that the server that I am connecting to actually is who it pretends to be. But – could I go around this? Could I use these certificates let’s say in an isolated environment? I mean, I have a valid certificate here for ibm.com. I have just saved it from Chrome like I did with my own certificate. Obviously I can’t fake IBM’s identity in the web because I don’t control their DNS servers. And I have no interest doing this. But - could I now let’s say define ibm.com to be my internal domain name here in my LAN with my local DNS Server, then set up a web server – still in my local LAN and set up a host called www, then browse to that internal web server – would I get one of these nasty security warnings or would I get a certificate for free? You may have guessed the answer – this is not possible. But why? And how?
Well, this is how certificates are supposed to work. Only the owner can use them but anyone and everyone can verify them. Just like Photo ID in the real world. Let’s go back to XCA. The onemarcfifty.com certificate has been signed by the R3 certificate. That’s why the R3 appears green here in the signature field. And there is a public key stored inside the onemarcfifty certificate that allows me to verify that signature coming from the private key of the R3 certificate. Watch what happens when I delete the R3 certificate. I can still see the public key but I can’t verify the signature because the signing certificate is not there any more. Let me quickly explain how private and public keys work.
(How private/public keys work)
In a nutshell, with algorithms like RSA we can do four or rather five things with private or public keys. We can encrypt, decrypt, sign and verify. And as a consequence, we can authenticate. The trick is now that you can only verify the signature that has been made with one key by using the other key. The same is true for encryption and decryption. You can only decrypt data with one key that had been encrypted with the other key. In real life we use the private key to sign something. That means that only the public key can verify the signature. One person can sign and everyone can verify. The other way round everyone can encrypt with the public key and only the owner of the private key can decrypt. How is that possible? Let’s do a little bit of math. Don’t worry. It will not get too complicated.
(a tiny bit of math)
In fact, we would need an operation that can easily be one in one direction but is impossible or let’s say hard to do in the other direction. Let me give you one example. If I was to calculate let’s say 3 times 7 times 17 times 23 – that’s quite easy to do. The result is 8211. The numbers that I have used for this multiplication are prime numbers. Now if I would ask you to do this the other way round – so if I would ask you to tell me which prime numbers I have multiplied in order to get to 8211, then this is much much harder to do. If the calculation in one direction would need let’s say 4 CPU cycles, then the reverse calculation could use hundreds or maybe even thousands of cycles. It would take much much more time. Now of course, this is feasible with 8211. But what if I had not used 7 or 17 as prime numbers but really huge prime numbers such as 2 to the power of 255 minus 19? That’s a number with 77 digits!!! Virtually impossible. It would take the fastest computer on this planet thousands of years to calculate it. Maybe it could go faster if it had a huge hash table with all known prime numbers and a really big number of pre-calculated hashes. The algorithms that are used for public and private keys are of course much more complex than this – but I think you get the idea. In a nutshell, it is not impossible to let’s say calculate or find a private key if you only have the public key. It would just take incredibly long. We therefore say that the cipher is computationally secure.
Back to the certificates.
In order to use a certificate for all operations, I would need the public key (which I have, because it’s stored inside the certificate) and the private key (which I don’t have – in my ibm.com example, only ibm have it). That’s the reason why I can’t fake their identity. I can’t build a chain of trust without having both keys. So when and how would a public CA issue a certificate to me? To Marc from Germany? They would need to be sure that I am who I pretend to be – or rather that my server is who it pretends to be.
(How are Let’s Encrypt doing it?)
How are ISRG and Let’s Encrypt doing it? I mean, all the browser vendors are trusting the Let’s Encrypt certificates. And they have issued a certificate to me – how would they make sure that they can stand up to Firefox, Chrome, Edge or whoever and say “Trust us, we have verified Marc’s identity” – I mean, I have never spoken to them? Well, Let’s encrypt are using DNS for this. They don’t actually have to verify my personal identity, just the one of my server. The thought process is as follows. If a host – let’s say www.onemarcfifty.com requests a certificate from ISRG, then all they’d have to do is do a DNS lookup of that address. And if that DNS lookup points to the host that made the request then it’s authentic. Because the person initiating the request needs to have control over the domain’s DNS entries. That process can easily be automated. And they do provide me with a signed certificate and a private key for that single specific identity. If I am able to put some arbitrary text entry on demand into my DNS, then they are even willing to issue a so called wild card certificate to me. That means that I could request a certificate that would be valid for any host in the onemarcfifty.com domain. For free. This type of authentication is called Domain validation (DV) – I authenticate by proof of control over a domain, in my case onemarcfifty.com.
(Let’s create our own CA and certificates)
So far so good. We now know that there are different types of certificates. Some of these are called a Certificate Authority or short “CA”. A CA can sign other certificates or even other CA’s just like ISRG signed the R3 CA which in turn signed my onemarcfifty.com Server certificate. So how would I proceed if I wanted to use certificates for my servers at home? How could I generate my own certificates? Well, basically I have three alternatives here. Either I use a paid or free service that issues Server or wildcard certificates to me, such as Let’s Encrypt. Or – I ask one of the root CA’s to sign my own CA which in turn I could then use to sign as many certificates as I please. A so called “vanity CA”. Or – that’s alternative 3 – I use my own self-signed certificate. Each of these mechanisms has advantages and challenges. Let’s Encrypt and the like require me to use an agent and to be in control of DNS. Not a big thing. There’s many videos on that.
And in the next episode I’ll show you a couple of tricks around that such as automating wildcard certificate requests with Ansible or using Let's Encrypt certificates for your internal network.
The limitation of these services is usually that you only get server certificates. The second method – the vanity CA – has one big challenge: The price. I checked on multiple provider pages for pricing and all of them just stated “ask for quote” or “on demand” – that doesn’t sound cheap. Also – you would presumably have to provide a lot of security procedures in order to prove to them that you don’t issue certificates for just anybody. Remember – if your CA is signed by a trusted CA, then you become part of the globally recognized chain of trust and – as such – a high value target. I wouldn’t actually want that. So what’s left? Actually, we can easily create our own Certificate Authority or CA. A self-signed CA. Just no one else would trust it – so probably not a good solution for a public web server but there are valid use cases for this.
(When and how to use self-signed CA)
A CA is really just a normal certificate which has a couple of specific attributes. Here in XCA we can add them down here with the default CA template. That does set a couple of things here in the Extensions tab and the Key usage tab. As a minimum I need to give the CA a name and a common name. I strongly advice to also assign it an organizationalName and a country code. And I need to generate a private key. Cool, so now I have a CA that is – of course – trusted by no one on this planet.
However, we can now generate more certificates. Here we see the first difference. Now I can not only create self-signed certificates, but I can actually sign this certificate with my CA. Let me first apply the TLS Server template here. Again – I need to give it a name and a common name, preferrably an organization and a country code and I need to generate a key for it. I’ll use a server or router name in my sandbox as the common name here because I want to use it for that server. Let me export that server certificate in pem format, that means just the plain certificate. Without the key. There are other formats available. Some of these do contain the key and others don’t. The web server that I want to use the certificate with needs both in separate files. So next, I’ll export the corresponding key – again in pem text format.
That’s it – I can now use that certificate for that server. But in order to have my browser trust it, I need it to recognize my CA certificate. But no problem, I own it so I can just import it to my web browser. In XCA I export the CA certificate. We don’t need the key for this – just like we don’t have the key for the public ones. Then next – in my browser I go back to the certificate settings or info page and there I can import the new CA. My browser will now trust any certificate that has been issued by that CA.
So when can and when would we use self signed certificates? In general, we can use this mechanism when we have control over both parties, that means the client and the server. Just like I showed here. I control the server and the client (my web browser). Or – maybe I don’t trust the public CA’s for some reason. However, there are challenges when you have many clients. As your CA is not built into the browsers, you would need to import or roll out the ca certificate to all of them. But let’s have a look at options for that scenario in the next episode when we will see how we can use free Let’s encrypt certificates for servers for our local network and also more options for self-signed CA’s.
Just a last remark with regards to security, vulnerabilities and so on. You may have heard that the security gets better with longer keys. That’s true. Also there are different cipher and hashing algorithms that you can use. Some of them are compromised, in other words proven to be insecure. MD5 is an example for this. However – please keep in mind that most successful attacks against crypto do NOT break the underlying crypto primitives but rather get around them. In other words – why would I want to brute force your private key if I can easily steal it from your workstation? So TLS, Certificates, encryption, authentication – all these things do not only rely on a good algorithm and a long key. It is equally important to have a look at the whole process of generating and distributing keys and certificates and also how you manage your keys. In other words – the real challenge is not encryption or decryption but rather Key management. But more on this in later episodes.
Until then – like always – thank you so much for watching. Stay safe, stay healthy, bye for now.
