Kubernetes Secrets Replication with ESO

Kubernetes Secrets Replication with ESO

Kubernetes Secrets Replication with ESO

Replicating secrets across namespaces is a common challenge in Kubernetes environments, particularly when multiple applications require shared access to sensitive data like database credentials or API keys. While there are tools like Kyverno that can handle this, they often fall short in terms of synchronization and integration with external secret stores.

In this guide, we’ll demonstrate how to use the External Secrets Operator (ESO) to achieve seamless replication of secrets across namespaces. By leveraging ClusterSecretStore and ClusterExternalSecret, you can centralize secret management, maintain synchronization with external sources, and avoid manual intervention.

Why Use ESO for Replication in Kubernetes?

When dealing with secrets in Kubernetes, replication across namespaces can be tricky:

  • Manual Processes: Tools like Kyverno require the source secret to exist before replication begins.
  • No Real-Time Sync: Changes to the source secret aren’t automatically reflected in the replicated copies.
  • Complexity in Scaling: Managing replication policies for multiple namespaces can quickly become unmanageable.

External Secrets Operator solves these issues by directly syncing secrets from external providers and ensuring they are always up-to-date in target namespaces. It simplifies secrets replication while adhering to Kubernetes best practices.

Step-by-Step Guide

1. Prepare Your Environment

To get started, ensure the following:

  • Kubernetes cluster with External Secrets Operator installed.
  • Access to an external secrets provider (e.g., AWS Secrets Manager, GCP Secret Manager, etc.).

For this demonstration, we’ll replicate a secret from a source namespace across all other namespaces in the cluster.

2. Create a Source Namespace and Secret

Begin by creating a namespace (source-ns) and a source secret (source-secret):

apiVersion: v1
kind: Namespace
metadata:
  name: source-ns
---
apiVersion: v1
kind: Secret
metadata:
  name: source-secret
  namespace: source-ns
data:
  foo: YmFy  ## bar

3. Configure Service Account and Permissions

The External Secrets Operator needs permissions to read the source secret. Create a service account and the necessary RBAC roles:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: k8s-store
  namespace: source-ns
--- 
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: read-source-secret
  namespace: source-ns
rules:
- apiGroups: [""]
  resourceNames:
  - source-secret
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - authorization.k8s.io
  resources:
  - selfsubjectrulesreviews
  verbs:
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-source-secret
  namespace: source-ns
subjects:
- kind: ServiceAccount
  name: k8s-store
  namespace: source-ns
roleRef:
  kind: Role #this must be Role or ClusterRole
  name: read-source-secret
  apiGroup: rbac.authorization.k8s.io

4. Create a ClusterSecretStore

The ClusterSecretStore connects External Secrets Operator to the source namespace:

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: k8s-store
spec:
  provider:
    kubernetes:
      # with this, the store is able to pull only from `default` namespace
      remoteNamespace: source-ns
      server:
        url: "https://kubernetes.default" # Change accordingly to your setup
        caProvider:
          type: ConfigMap
          name: kube-root-ca.crt
          key: ca.crt
          namespace: source-ns
      auth:
        serviceAccount:
          name: "k8s-store"
          namespace: source-ns

5. Create a ClusterExternalSecret

The ClusterExternalSecret specifies the secret to replicate and its target namespaces:

apiVersion: external-secrets.io/v1beta1
kind: ClusterExternalSecret
metadata:
  name: "copy-secret"
spec:
  # The name to be used on the ExternalSecrets
  externalSecretName: "copy-secret-es"
  namespaceSelectors:
  - matchLabels: {} #deploy to every namespace
  refreshTime: "1h" #Cluster External Secret refresh Time
  externalSecretSpec:
    secretStoreRef:
      name: k8s-store
      kind: ClusterSecretStore
    refreshInterval: "10m" #External Secret Refresh Time
    target:
      name: copy-secret
      creationPolicy: 'Owner'
    dataFrom:
      - extract: 
          key: source-secret

6. Apply the Configurations and Verify

Apply all the manifests and check the status of the ExternalSecrets:

kubectl apply -f <your-manifest-files>
kubectl get es -A

default                 copy-secret-es   k8s-store   10m0s              SecretSynced   True
source-ns            copy-secret-es   k8s-store   10m0s              SecretSynced   True

And with it, every new namespace creation automatically triggers a new copy of that secret, effectively replicating the sensitive information across namespaces.

7. Automatic Secrets Replication for New Namespaces

With this setup, any new namespace created in the cluster will automatically receive the replicated secret, ensuring consistent access across your environment.

Conclusion

Managing secrets replication across namespaces in Kubernetes doesn’t have to be a challenge. By leveraging the External Secrets Operator, you can automate this process, ensure synchronization with external secret providers, and maintain a scalable and secure workflow. This approach not only simplifies secrets management but also ensures you’re following Kubernetes best practices for handling sensitive data. For more advanced use cases or enterprise-level features, explore our solutions at www.externalsecrets.com.

blog-image
Kubernetes Secrets Replication with ESO

Replicating secrets across namespaces is a common challenge in Kubernetes environments, particularly when multiple applications require shared access to sensitive data like database credentials or API keys. While there are tools like Kyverno that can handle this, they often fall short in terms of synchronization and integration with external secret stores. In this guide, we’ll demonstrate how to use the External Secrets Operator (ESO) to achieve seamless replication of secrets across namespaces.

blog-image
Just in time rotation is out!

Our Just-in-Time Rotation feature is now live, and you can explore it for free on our platform! We shared our plans in a recent blog post: Async Rotation and after gathering feedback and listening to requests from our community and partners, we officially launched this feature with a focus on real-world use cases. Many organizations face the challenge of waiting for data to sync within a refresh interval. Just-in-Time Rotation solves this by ensuring that updated secrets are available immediately upon changes in the source of truth or triggered by supported event sources.

blog-image
Isolating Secrets Management within Kubernetes Namespaces

Why should I care? This piece will be useful to you if you need to ensure that workloads in your cluster don’t have cluster-scoped access to resources, even if they are controllers that one would typically consider closer to the control plane than to individual workloads. You are probably looking for ways to configure external secrets management with namespace isolation or more specifically how to configure ESO (External Secrets Operator – external-secrets) through a namespaced approach.

Join us for effortless Secrets Management

Sign Up