Partner POV | Enterprise Secrets Management Explained: Best Practices, Challenges, and Tool Selection

This article was written and contributed by our partner, Thales.

Whether hosted in the cloud or on-premises, modern applications and integrations have accelerated the need for digital secrets. These secrets control data access when transferred between applications—sending information from a webpage, making a secure request to an API, accessing a cloud database, or countless other cases that modern enterprises encounter while pursuing digital transformation and increasing automation. However, for access control to be robust, businesses must effectively manage the respective secrets across their lifecycle and protect them from compromise. This is precisely what secrets management ensures.

What is secrets management?

Secrets management refers to tools and methods to securely store, access, and centrally manage the lifecycle of digital authentication credentials. This includes sensitive data such as passwords, encryption keys, APIs, tokens, and certificates. These secrets authenticate a user or machine to access applications or services within an organization's IT ecosystem.

By minimizing the role of humans in secrets management, organizations can avoid data breaches, identity theft, and other identity-related problems by using a methodical approach to prevent unauthorized access to sensitive data and systems.

Key concepts of secrets management

What is a secret?

The simplest definition of a secret is that of a digital authentication credential. In that sense, the most well-known example of a secret is a password. Secrets refers to private information vital to unlocking protected resources or sensitive information in tools, applications, containers, DevOps, and cloud-native environments.

The most common types of secrets include:

• Privileged account credentials
• Passwords
• TLS/SSL certificates
• SSH keys
• API keys
• Encryption keys

Although human-related passwords are the most well-known type of secret, non-human secrets are more ubiquitous and more challenging to manage and secure. With the proliferation of non-human entities, collectively named machines, including IoT devices, apps, APIs, containers, and microservices, machine secrets have skyrocketed, making their automated management necessary to limit the possibility and the impact of a compromise.

Fireside Chat: What is Secrets Management and Why Do You Need It?

The challenges of secrets management

Poor secrets management can expose organizations to many cyber risks. The Verizon 2023 Data Breach Investigations Report indicates that stolen or compromised credentials are the primary vector for attackers to break into the business's crown jewel, the data. Credentials (along with personal data) are the more sought-after assets during a breach; for good reasons, as they are the keys to our kingdom.

Although the human element is involved in 74% of data breaches, stolen credentials provide the necessary second step for the intruder to move laterally within the corporate network and escalate privileges. Over 80% of breaches regarding web-based applications can be attributed to stolen credentials. According to Verizon, over the past five years the use of stolen credentials has become the most popular entry point for breaches.

Part of the poor secrets management problem is the challenges organizations face in successfully implementing this process. The more complex an IT ecosystem and the more diverse and numerous the secrets, the harder it is to securely store, transfer, and track secrets.

Secrets sprawl

In addition to the common uses of secrets mentioned above, secrets are also embedded as hard-coded credentials in containerized applications, Robotic Process Automation (RPA) platforms, business-critical applications, including both internally developed and commercial off-the-shelf solutions (COTS), and the Continuous Integration/Continuous Deployment (CI/CD) pipeline. This creates a much more pervasive occurrence of secrets that is referred to as the secrets sprawl.

Lack of comprehensive visibility

Secrets sprawl makes managing secrets more difficult. In certain enterprises, the number of SSH keys alone could reach millions. Decentralized methods, where administrators, developers, and other team members all manage their secrets individually, if at all, suffer from this drawback. Security holes and auditing difficulties are certain to exist without unified visibility that spans all IT ecosystems.

Hardcoded default secrets

Hardcoded default credentials are frequently distributed and deployed with applications and IoT devices, and they are straightforward to crack using scanning tools and dictionary-style or guessing attacks. Secrets are also routinely hardcoded in scripts or files for DevOps technologies, endangering the security of the entire automated process.

DevOps tools

DevOps environments are where the challenges of managing secrets are amplified. DevOps teams typically leverage dozens of orchestrations, configuration management, and other tools and technologies like Ansible or Docker, relying on automation and other scripts that require secrets to work. These secrets should all be managed according to best security practices, including credential rotation, time/activity-limited access, auditing, and more.

Manual secrets management processes

Manual processes are known to be cumbersome and error prone. Poor management means secrets are not likely to remain secret, opening the opportunity for breaches. Generally, the more manual secrets management processes equate to a higher likelihood of security gaps and malpractices.

The need for enterprise secrets management solutions and best practices

Enterprise secrets management tools and best practices can resolve these challenges and prevent unauthorized access to sensitive data, reducing the risk of data breaches and unauthorized manipulation of sensitive corporate and personal data. These can have disastrous consequences for an organization, including direct financial loss, reputational damage, legal exposure, and regulatory fines.

Businesses use secrets management solutions to manage their IT ecosystem's secrets centrally. These tools and established best practices reduce the risks associated with poor and manual secrets management, such as hardcoding secrets into scripts, using default passwords, manually sharing passwords, and failing to rotate credentials. Secrets management tools replace manual processes and provide centralized visibility, monitoring, and management for secrets across an organization.

The foundation of secrets management is identifying all the types of secrets within the organization, where they are located, and continuously maintaining an inventory. Besides creating the inventory, it is equally essential to identify all the secrets management use cases, such as to secure CI/CD pipelines, containers, ephemeral environments, IoT devices, RPA processes, and internally developed applications.

This comprehensive inventory can help organizations monitor and audit secrets more effectively for security and compliance purposes. Auditing is an essential part of secrets management due to the nature of the process. At a minimum, organizations should monitor and audit who requested a secret and for what system and role, when the secret was used and by whom, when the secret has expired, and more.

Secrets management best practices

The following are the pillars of an effective and efficient secrets management process:

• Create a comprehensive secrets inventory. This is the foundation of secrets management. You can't protect something you don't know. Take the time to cover all the secrets used throughout your organization.
• Create a comprehensive secrets management policy. The policy should set strict rules governing the lifecycle of secrets (strength, expiration, rotation, revocation) while prohibiting the use of default or hardcoded secrets.
• Automate, automate, automate. Remove the human element from the secrets management process, and rely on automation platforms to create, manage, distribute, and maintain secrets.
• Encrypt data using a Key Management solution. A key management solution to encrypt data at rest, in transit, and in use stores and manages keys and provides them automatically when data needs to be encrypted or decrypted. It also makes it possible to encrypt each dataset or resource with a different encryption key, controlling access more granularly.
• Rotate secrets frequently. Secrets should be changed regularly to limit the impact of a potential compromise. If a secret remains unchanged for a long time, more users and systems gain access to it and can potentially compromise it. When using a secrets management tool, ensure using its rotation functionality and set rotation to a sufficiently high frequency.
• Segregate duties and separate data from secrets. It is always a best practice to segregate duties and keep secrets separate from data to limit the potential of a breach affecting both assets.
• Manage privileges. It is important to follow the principle of least privilege, in which a user or application is only granted privileges if necessary to perform its role. When access is no longer needed, it should be revoked. Privileged sessions should be carefully monitored to improve monitoring and accountability.
• Detect unauthorized access. Breaches are a matter of when and not if. Ensure having a robust process for monitoring and identifying unauthorized access and establish an incident response process to mitigate breaches and reduce the impact on the organization.
• Train your employees. Training your employees across all departments on secrets management best practices is essential. It is the best way to enforce the secrets management policy and best practices and limit the possibility of costly mistakes and oversights.

Choosing a secrets management tool

Using a dedicated secrets management solution is highly recommended. Most cloud providers have at least one service that offers secret management. For example, for AWS the recommended solution is AWS secret manager, Google recommends using its secret manager, and for Azure the recommended service is Key Vault. In a multi cloud environment that most customers find themselves in, there is need for a secrets management solution that spans cloud providers.

Businesses should consider using a cloud-neutral secrets management solution if they implement a multi-cloud strategy. A cloud-neutral option will allow them to use one secrets management solution across all cloud platforms (and possibly also on-premises).

Picking the right secrets management solution

The following set of actions must be kept in mind while selecting a secrets management solution:

• Do market research and identify the types of secrets management tools available, including open-source and commercial solutions.
• Consider the organization's specific needs and use cases when selecting a secrets management tool. Consider if the identified tools satisfy current and future business needs. As your organization expands to more environments and regions, it is essential for the solution to include scalable integration capabilities with support for a wide variety of plugins.
• Evaluate the level of security and encryption provided by the tool. Do the solutions provide options for secrets rotation, just-in-time access, and encryption with adequate resistance against quantum threats?
• Consider the ease of integration with other tools and systems in the organization's environment. The right platform should allow seamless cross-platform, cross-environment workflows. The selected solution should be completely agnostic and work in cloud and legacy IT environments. In addition, it must support integrations with the most common cloud platforms, such as Kubernetes and Docker.
• Evaluate the level of support and documentation provided by the tool's vendor or community.
• Consider the cost and budget constraints when selecting a secrets management tool. Hidden costs may increase expenditures, making the solution not a cost-effective option.
• Keep in mind any regulatory or compliance requirements that may impact the selection of a secrets management tool. Data sovereignty and localization requirements may limit your options, and you should look for solutions that provide compliance per jurisdiction.
• Consider conducting a proof of concept or trial of the tool before making a final decision. This is essential to ensure the tool does what it promises.