What is TLS encryption and how does it work?

TLS, which is short for Transport Layer Security, is an email security protocol based on cryptography. It facilitates the end-to-end security of data transmitted between applications over the Internet. Most people know it as the padlock icon that appears in web browsers when a secure session is established. But there is more to it—it’s also used in emails, file transfers, video and audio conferencing, instant messaging, and voice-over IP.  The overall aim of the TLS is to add an extra layer of security, preventing threat actors from hijacking connections between internet-enabled devices. It lets you know whether the person you are communicating with is actually who they are claiming to be. 

TLS was proposed by the Internet Engineering Task Force (IETF), and its premier version was released in 1999. We currently use the TLS 1.3 version, which was published in 2018. 

What does TLS do?

Before understanding the TLS process, you should know that email is usually exchanged using SMTP to send and relay messages. Other protocols like IMAP and POP3 are used to retrieve emails from a server. While SMTP takes care of the delivery of email from the sender’s server to the receiver’s, IMAP/POP3 is used by the recipient’s client to fetch the email from the mail server.

By default, SMTP, POP3, and IMAP do not encrypt email traffic, which means that data, including the contents of emails, can be intercepted and read by malicious actors. TLS helps address this by encrypting the connection.

There are two ways to secure emails using TLS-

STARTTLS

STARTTLS is a command used to upgrade an existing plaintext connection (unencrypted) to an encrypted one using TLS. It works with both SMTP (for sending emails) and IMAP/POP3 (for receiving emails). 

The process begins when the client connects to the server over a plain, encrypted connection. Then, the client sends the ‘STARTTLS’ command to signal the intent to upgrade the connection to a secure one. If the server responds positively, the TLS handshake is established, creating a secure and encrypted session. After the handshake process is completed, communication continues over the encrypted connection, ensuring the data is protected. 

Implicit TLS

If the process is done using Implicit TLS, then the connection starts as encrypted from the beginning, and there is no need for an upgrade using the STARTTLS command. In this method, the client directly initiates a connection over TLS, and the communication remains encrypted throughout the session.

TLS handshake process

Regardless of whether STARTTLS or Implicit TLS is used, the handshake process remains the same. 

Step 1: Client Hello

The client (mail server or email client) initiates a connection with the server by sending a Client Hello message. This message includes a list of supported encryption algorithms (cipher suites) such as AES, RSA, ECDHE, etc., supported TLS versions (e.g., TLS 1.2, TLS 1.3), and random data for generating keys.

Step 2: Server Hello

The server responds with a Server Hello message, which contains:

• The chosen cipher suite (based on what the client supports)
• The server’s digital certificate (public key), issued by a Certificate Authority (CA) to prove its identity
• Random data for generating keys
• In some cases, the server may request the client’s certificate (for mutual authentication, although this is rare in email exchanges).

Step 3: Certificate verification

The client verifies the server’s certificate by evaluating whether it’s valid and trusted. This is done by verifying the signature with the CA’s public key. If it matches, the connection continues; otherwise, it fails.

Step 4: Key exchange and session key generation

To generate a session key, the client and server use the random data exchanged during the Hello phase and the server’s public key. In older versions like TLS 1.2, the client encrypts a pre-master secret with the server’s public key and sends it to the server. 

However, the recent TLS 1.3 version uses a faster and more secure key exchange mechanism, which ensures forward secrecy. In simpler words, it ensures that even if the private key is compromised in the future, past communications will remain secure. 

Step 5: Handshake completion

Once the session key is established, the client and server send a message confirming that the handshake is complete. The communication is now encrypted, and both parties can exchange email data securely.

Difference between SSL and TLS

SSL and TLS offer the same purpose, which is why most people use them interchangeably; however, that’s wrong because there are differences between the two security protocols. 

Final words

TLS should be combined with SPF, DKIM, and DMARC to provide a robust defense against email-based menaces. Together, these technologies facilitate confidentially and authentication while improving the deliverability rate and domain reputation. To get started with DMARC, reach out to us.