Shared Data Protection
Data Protection in a widely deployed server ecosystem.
- Whenever data is protected with encryption, all sites that need to access that data must have access to a decryption process that is trusted with the decryption keying material.
- Different apps have different security profiles. DPAPI was designed to support very high levels of security.
- Whenever a secret is widely shared, it should be treated as public knowledge as every process that can access the secret needs to as secure as the information protected by the secret.
- Secrets should never be accessible by any process that is accessed by any untrusted process. For example a Web Site that is accessed by the public internet should never have access to secrets.
- Since Web Sites may access data that is protected with encryption by secret keys, the process that does the decryption should not be in the same process, or security domain, as the Web Site.
- One reasonably secure approach to the problem would be to have a user-defined encryption key (either static or derived from passphrase, initialization vector (iv), etc) which should be used by all servers on your farm. What needs to be done at application installation (on each server) define this key and store it in secure form, so that only a secure application can retrieve it.
Windows introduced in NT5 (that is, any release after NT4) the Data Protection API or DPAPI (pronounced d-pappy) to protect secrets on that computer or an any computer that shared a Kerberos logon account that the secrets were bound to.
- If the application is a Windows service (or some Windows app which can run with loaded user profile), this key can be encrypted (and stored in the registry or app.config file) using DPAPI with user store of the account under which the app will run. This, of course assumes that the app that decrypts the data and the app used the encrypt key run under the same user account. Still the key (or keying material, such as iv, etc) must be somewhere to make sure it can be redefined on another system. (Note that symmetric key encryption applies to the above assertion. With public key encryption there is only a need for all servers that can decrypt the data share the private keying material.)
- If the app is an ASP.NET app (or some app which cannot run with loaded user profile, such as Web service), you can only use DPAPI with machine key. The same logic/process, can be used, but it is not secure, because anyone who gets access to the server will be able to decrypt data (in the previous case, the user would need to know the password of the user account, which is unlikely). While some may argue that their servers are unhackable, life proves that even most guarded systems can be broken into (due to viruses, app vulnerabilities, admin/user mistakes, etc). Anyway, if this is an acceptable risk to the enterprise (and it may be depending on the value of the data, support infrastructure, etc), go ahead and use it. Check this article; it may offer you some ideas and relevant references: 
- A better solution is to run the decryption process under a different account. It is easier to make that account system wide and be responsible for key distribution.
In any case, under no cirstances should you encrypt data using DPAPI (with either user or machine generated key) if that data is needed for an Enterprise purpose. If your server gets rebuilt (or user account is changed), data will be gone and there is nothing anyone can do to restore it since there is no way to generate the same key.
While Unix derivatives never introduced anything like DPAPI, it has had the ability all along to have inter-process communications. Just provision a process running with high security privileges that can perform the decryption step. This should be an altogether different project than any web facing project with its own secure store and deployment process.
- Aaron Margosis+1 DPAPI in a Load Balanced Environment - ASP.NET Security http://justskins.com/forums/dpapi-in-a-load-64340.html
- "How to: Use Data Protection" (2017-03-28) https://docs.microsoft.com/en-us/dotnet/standard/security/how-to-use-data-protection
Other Source Material
- A discussion about DPAPI use in Identity Server 4.
- Keep High-Value Assets in a Stronghold, How Can You Protect Your Assets? A description of the onion model.