Encrypting SQL Server Database: Transparent Data Encryption (TDE)

Transparent Data Encryption (TDE), a built-in feature for encrypting data at rest. TDE protects the physical media that hold the data associated with a user database, including the data and log files and any backups or snapshots. Encrypting data at rest can help prevent those with malicious intent from being able to read the data should they manage to access the files.

Windows Data Protection API (DPAPI) sits at the top of the hierarchy and is used to encrypt the service master key (SMK), a symmetric key that resides in the master database. SQL Server creates the SMK the first time the instance is started. You can use the key to encrypt credentials, linked server passwords, and the database master keys (DMKs) residing in different databases.

So to implement TDE on a user database, we must take the following steps:

1. Create the DMK in the master database, if it doesn’t already exist.
2. Create a certificate in the master database for securing the DEK.
3. Create the DEK in the user database to be encrypted.
4. Enable TDE on the user database.

Let's setup a test database such as the one shown in the following T-SQL script:

 

USE master;

GO

 

CREATE DATABASE EmpData2;

GO

 

USE EmpData2;

GO

 

CREATE TABLE EmpInfo(

  EmpID INT PRIMARY KEY,

  NatID NVARCHAR(15) NOT NULL,

  LoginID NVARCHAR(256) NOT NULL);

GO

 

INSERT INTO EmpInfo(EmpID, NatID, LoginID)

SELECT BusinessEntityID, NationalIDNumber, LoginID

FROM AdventureWorks2014.HumanResources.Employee

WHERE NationalIDNumber IS NOT NULL;

1. CREATE THE DMK
To create the DMK that will support a TDE-enabled database, you take the same steps you take when creating the DMK to support column-level encryption, except for one important difference. You must create the key in the master database, as shown in the following T-SQL code:

USE master;

GO

 

CREATE MASTER KEY

ENCRYPTION BY PASSWORD = 'samimalik123';

 

To verify that the DMK has been created, we can query the sys.symmetric_keys catalog view:

SELECT name KeyName,

  symmetric_key_id KeyID,

  key_length KeyLength,

  algorithm_desc KeyAlgorithm

FROM sys.symmetric_keys;

The SELECT statement returns the results shown in the following table.
 

KeyName	KeyID	KeyLength	KeyAlgorithm
##MS_DatabaseMasterKey##	101	256	AES_256
##MS_ServiceMasterKey##	102	256	AES_256

 

Notice that the results include both the DMK and SMK. As already noted, SQL Server creates the SMK in the master database automatically. As you can see, the two keys are based on the 256-bit AES encryption algorithm.

 

2.  CREATE THE CERTIFICATE

 The next step is to create a certificate in the master database using a CREATE CERTIFICATE statement. In SQL Server, a certificate is a digitally signed, database-level securable that binds the public and private keys.
To keep things simple, we’ll create a self-signed certificate, which is automatically protected by the DMK. Normally, a certificate authority (CA) would issue and sign the certificate, which we would then incorporate into our encryption infrastructure, but a self-signed certificate can be handy for developing and testing, as well as checking out functionality like we’re doing here.
To create a self-signed certificate, we need only provide a name for the certificate and a WITH SUBJECT clause, as shown in the following statement:

 

CREATE CERTIFICATE TdeCert

WITH SUBJECT = 'TDE certificate';

The WITH SUBJECT clause supposedly specifies the issuer name; however, it can be just about any value, although a relevant description is normally the best option. In this case, I’ve gone with TDE certificate.
Note that, in addition to self-signed certificates, the CREATE CERTIFICATE statement lets us define a certificate based on a certificate file as well as retrieve the private key from a file or use a password to encrypt the certificate.
After we run the CREATE CERTIFICATE statement, we can verify that the certificate has been created by querying the sys.certificates catalog view:

SELECT name CertName,

  certificate_id CertID,

  pvt_key_encryption_type_desc EncryptType,

  issuer_name Issuer

FROM sys.certificates

WHERE issuer_name = 'TDE certificate';

 

the SELECT statement returned the results shown in the following table. 

 

CertName	CertID	EncryptType	Issuer
TdeCert	258	ENCRYPTED_BY_MASTER_KEY	TDE certificate

 

 The value in the EncryptType column is ENCRYPTED_BY_MASTER_KEY, which confirms that SQL Server has used the DMK to encrypt the certificate.

 

3. CREATE THE DEK

Now we switch over to our EmpData2 database to create the DEK, the next level down our hierarchy. When we create the DEK, we must specify the algorithm to use for the encryption key and the certificate to use to encrypt the DEK. Starting with SQL Server 2016, all algorithms have been deprecated except 128-bit AES, 192-bit AES, and 256-bit AES. (The higher the number of bits, the stronger the algorithm.)
To create the DEK, we can use a CREATE DATABASE ENCRYPTION KEY statement, as shown in the following example:

USE EmpData2;

GO

 

CREATE DATABASE ENCRYPTION KEY

WITH ALGORITHM = AES_256

ENCRYPTION BY SERVER CERTIFICATE TdeCert;

 

In this case, we’ve specified the 256-bit AES algorithm and the TdeCert certificate we created in the previous step. When you run the statement, you should receive the following warning.

 

Warning: The certificate used for encrypting the database encryption key has not been backed up. You should immediately back up the certificate and the private key associated with the certificate. If the certificate ever becomes unavailable or if you must restore or attach the database on another server, you must have backups of both the certificate and the private key or you will not be able to open the database.

 

This is an important message and one you should heed. We’ll discuss backing up your keys and certificates later in the article, but know that it is something you should be doing whenever you’re using them as part of your encryption process.
Now let’s return to our DEK. Once we’ve created the key, we can verify its existence by querying the sys.dm_database_encryption_keys dynamic management view:

SELECT DB_NAME(database_id) DbName,

  encryption_state EncryptState,

  key_algorithm KeyAlgorithm,

  key_length KeyLength,

  encryptor_type EncryptType

FROM sys.dm_database_encryption_keys;

 

The sys.dm_database_encryption_keys view returns details about a database’s encryption state and its associated DEKs. The following table shows the results returned by our SELECT statement.

DbName	EncryptState	KeyAlgorithm	KeyLength	EncryptType
EmpData2	1	AES	256	CERTIFICATE

Notice that the EncryptType column has a value of CERTIFICATE, which confirms that a certificate was used to encrypt the DEK.
Also notice that the EncryptState column shows a value of 1. This indicates that the database is in an unencrypted state. According to SQL Server documentation, the column can display any one of the values described in the following table.
 

Value	Description
0	No database encryption key present, no encryption
1	Unencrypted
2	Encryption in progress
3	Encrypted
4	Key change in progress
5	Decryption in progress
6	The certificate or asymmetric key encrypting the DEK is being changed

 

 4. ENABLE TDE ON THE USER DATABASE
We now have all the pieces in place to enable TDE on the EmpData2 database. The only step left is to turn encryption on.
Before we do that, I want to point out that there are many considerations to take into account before actually enabling TDE. For example, if any filegroups associated with the database are set as read-only, the encryption operation will fail. You’ll also run up against a number of restrictions when trying to implement TDE, such as not being able to drop a database during the initial encryption process.
Before you implement encryption on anything other than a test database in a test environment, I highly recommend that you review the MSDN article Transparent Data Encryption (TDE), which explains the various considerations and restrictions to take into account before implementing TDE.
With that in mind, let’s return to the matter at hand, which is to enable TDE on the EmpData2 database. To do so, we need only run a simple ALTER DATABASE statement that sets encryption on, as shown in the following example:

ALTER DATABASE EmpData2

SET ENCRYPTION ON;

 

 That’s all there is to it. Because our database is so small, the encryption process will be very quick. Not surprisingly, the larger the database, the longer this process will take.
If we again query the sys.dm_database_encryption_keys view, we’ll get the results shown in the following table, which verify that the EncryptState value is now 3.

DbName	EncryptState	KeyAlgorithm	KeyLength	EncryptType
tempdb	3	AES	256	ASYMMETRIC KEY
EmpData2	3	AES	256	CERTIFICATE

The results also show something else that’s very important to note—the addition of a row for the tempdb database. When you implement TDE on any user table, SQL Server also encrypts the tempdb database.
If you consider the logic behind this, you can see why Microsoft has taken this step. The database contains such items as temporary user objects, internal objects, and row versions, any of which can expose sensitive data. The downside to this is that unencrypted databases can take a performance hit, although Microsoft claims that the impact is minimal.
This issue aside, as long as our certificate and keys are in place, we can query the TDE-encrypted database just like we did before we enabled TDE. For example, we can run the following SELECT statement against the EmpInfo table:

SELECT TOP 5 * FROM EmpInfo;

 

Notice that we do not have to take any special steps with our query like we do with column-level encryption. We simply run the query as before, a fact that should make application developers happy. The following table shows the results returned by our SELECT statement.

EmpID	NatID	LoginID
1	295847284	adventure-works\ken0
2	245797967	adventure-works\terri0
3	509647174	adventure-works\roberto0
4	112457891	adventure-works\rob0
5	695256908	adventure-works\gail0

As you can see, we’re getting exactly the results we would expect. From the user/application perspective, it’s business as usual.

5. DISABLE TDE ON THE USER DATABASE
At some point, you might decide that you want to disable encryption on a user database. The process is as simple as enabling it. You again run an ALTER DATABASE statement, only this time turning off the encryption, as shown in the following example:

 

ALTER DATABASE EmpData2

SET ENCRYPTION OFF;

 

You can verify that encryption has been disabled by again querying the sys.dm_database_encryption_keys dynamic management view, which now returns the results shown in the following table:

DbName	EncryptState	KeyAlgorithm	KeyLength	EncryptKey
tempdb	3	AES	256	ASYMMETRIC KEY
EmpData2	1	AES	256	CERTIFICATE

As you can see, the EncryptState value for the EmpData2 database is now 1, indicating that it is in an unencrypted state. But notice that the tempdb database is still encrypted. As it turns out, the database will stay encrypted until it is re-created, which occurs whenever the SQL Server service restarts.

For a development or test environment, restarting the service might not be a big deal, but restarting a production instance is an entirely different matter. One more reason to give careful consideration to implementing TDE.
In the meantime, if you do have control over your instance of SQL Server, you can restart the service to see for yourself what happens with the tempdb database. From there, you can again query the sys.dm_database_encryption_keys view, which should return the results shown in the following table.

DbName	EncryptState	KeyAlgorithm	KeyLength	EncryptKey
EmpData2	1	AES	256	CERTIFICATE

As you can see, the tempdb database is no longer included in the results because the database has not been encrypted or subjected to TDE.
If you disable TDE on your database, you’re also free to drop the DMK, certificate, and DEK, using the DROP MASTER KEY, DROP CERTIFICATE, and DROP DATABASE ENCRYPTION KEY statements, respectively. Or you can re-enable TDE on the user database at any point. Just keep in mind the impact on the tempdb database.

6. BACKUP THE CERTIFICATE AND KEYS
As already noted, you should back up your certificates and keys, preferably right after you create them. This is also true for the SMK, before you start relying on it to protect your DMKs.
To back up the SMK, you can use a BACKUP SERVICE MASTER KEY statement, as shown in the following example:

Use master;

GO

 

BACKUP SERVICE MASTER KEY

TO FILE = 'C:\DataFiles\MsSqlServer\SvcMasterKey.key'

ENCRYPTION BY PASSWORD = 'samimalik123';

The statement itself is fairly straightforward. You must provide the full path for the backup file and a password for encrypting the key in that file. One thing to note, however, is that not all the examples in the Microsoft documentation clearly demonstrate that you must provide a full path, including the file name and its extension. Without these, you will receive an error.
It’s also worth noting that the BACKUP SERVICE MASTER KEY statement includes no logic for what to do when the file already exists. If it does exist, you’ll again receive an error.
Backing up the DMK works much the same way, except that you use a BACKUP MASTER KEY statement:

BACKUP MASTER KEY

TO FILE = 'C:\DataFiles\MsSqlServer\DbMasterKey.key'

ENCRYPTION BY PASSWORD = 'samimalik123'

 gain, you must provide the full path and file name, along with a password. In addition, the file cannot already exist. Backing up a certificate is a little different because you want to be sure to explicitly back up the private key along with the certificate. For that, you must use a BACKUP CERTIFICATE statement that includes the WITH PRIVATE KEY clause, as shown in the following example:

BACKUP CERTIFICATE TdeCert

TO FILE = 'C:\DataFiles\MsSqlServer\TdeCert.cer'

WITH PRIVATE KEY(

  FILE = 'C:\DataFiles\MsSqlServer\TdeCert.key',

  ENCRYPTION BY PASSWORD = 'pw1234!'

);

 

In this case, we’re generating a file for both the certificate and the private key, as well as providing a password for the private key. That’s all there is to backing up the certificate and keys. Of course, you should store your backup files in a remote locate separate from the database files, ensuring that they’re protected from any sort of mischief or recklessness.

Under the right circumstances, TDE can be a useful addition to your data protection strategies. However, TDE protects only data at rest, such as the data and log files. It does not protect data in memory or data transmitted between an application and SQL Server. As a result, your database is still vulnerable to such risks as SQL injection attacks or hijacked administrative permissions. Once data has been unencrypted for use, it is just as vulnerable as it has always been.
If you decide to implement TDE, you should do so only as one piece of a much more comprehensive security strategy that takes into account all possible risks to your data. You might also consider alternatives to TDE, such as the new Always Encrypted feature introduced in SQL Server 2016 (a topic we’ll cover in a future article).
When Microsoft introduced TDE in SQL Server 2008, it represented an important step in data protection, but it was only one step and was never meant to be the sole means of safeguarding your data. Be sure to keep that in mind when working with TDE.