Shared Data Protection

Full Title

Data Protection in a widely deployed server ecosystem.

Context

• Whenever data is protected with encryption, all sites that need to access that data must have access to a decryption process that is trusted with the decryption keying material.
• Different apps have different security profiles. In the Windows operation system DPAPI was designed to support very high levels of security within an Entity that had control of the entire computing environment.
• This page assumes that data protection is not subject to governmental seizures. See the wiki page International Data Protection for the problem of data held in other countries.

Problems

• Whenever a secret is widely shared, it should be treated as public knowledge as every process that can access the secret needs to as secure as the information protected by the secret.
• Secrets should never be accessible by any process that is accessed by any untrusted process. For example a Web Site that is accessed by the public internet should never have access to secrets.
• Since Web Sites may access data that is protected with encryption by secret keys, the process that does the decryption should not be in the same process, or security domain, as the Web Site.

Solution

• One reasonably secure approach to the problem would be to have a user-defined encryption key (either static or derived from passphrase, initialization vector (iv), etc) which should be used by all servers on your farm. What needs to be done at application installation (on each server) define this key and store it in secure form, so that only a secure application can retrieve it.

Windows

Windows introduced in NT5 (that is, any release after NT4) the Data Protection API or DPAPI (pronounced d-pappy) to protect secrets on that computer or an any computer that shared a Kerberos logon account that the secrets were bound to.

• If the application is a Windows service (or some Windows app which can run with loaded user profile), this key can be encrypted (and stored in the registry or app.config file) using DPAPI with user store of the account under which the app will run. This, of course assumes that the app that decrypts the data and the app used the encrypt key run under the same user account. Still the key (or keying material, such as iv, etc) must be somewhere to make sure it can be redefined on another system. (Note that symmetric key encryption applies to the above assertion. With public key encryption there is only a need for all servers that can decrypt the data share the private keying material.)
• If the app is an ASP.NET app (or some app which cannot run with loaded user profile, such as Web service), you can only use DPAPI with machine key. The same logic/process, can be used, but it is not secure, because anyone who gets access to the server will be able to decrypt data (in the previous case, the user would need to know the password of the user account, which is unlikely). While some may argue that their servers are unhackable, life proves that even most guarded systems can be broken into (due to viruses, app vulnerabilities, admin/user mistakes, etc). Anyway, if this is an acceptable risk to the enterprise (and it may be depending on the value of the data, support infrastructure, etc), go ahead and use it. Check this article; it may offer you some ideas and relevant references:
• A better solution is to run the decryption process under a different account. It is easier to make that account system wide and be responsible for key distribution.

In any case, under no cirstances should you encrypt data using DPAPI (with either user or machine generated key) if that data is needed for an Enterprise purpose. If your server gets rebuilt (or user account is changed), data will be gone and there is nothing anyone can do to restore it since there is no way to generate the same key.

Unix

While Unix derivatives never introduced anything like DPAPI, it has had the ability all along to have inter-process communications. Just provision a process running with high security privileges that can perform the decryption step. This should be an altogether different project than any web facing project with its own secure store and deployment process.

Server Farms

When it is not clear which platform (Windows or Unix in particular) will be processing data that needs access to user secrets, one solution would be a central server that can dole out secrets to process that have the privileges to request them. One very simple way to bring up such a service is to place the secrets in a private git repository that can only be accessed by process with the desired credentials.

Create File with just Password

Using PowerShell

Set-Content test.txt ([byte[]][char[]] "test") -Encoding Byte

References

Other Source Material

• A discussion about DPAPI use in Identity Server 4.
• Keep High-Value Assets in a Stronghold, How Can You Protect Your Assets? A description of the onion model.