---
menu:
  workshop:
    parent: Secrets Management
title: ESO and Vault
weight: 61
---

:toc:
include::modules/comm-attributes.adoc[]
:imagesdir: /images/workshop

[#esoVault]
== ESO and Vault

IMPORTANT: The Validated Patterns team settled on using **references to secrets**

We currently implement GitOps + Secrets via link:https://external-secrets.io/v0.7.0/[External Secrets Operator] and link:https://www.vaultproject.io/[HashiCorp Vault]

The main reasons for this choice at the time were the following:

* Sealed Secret approach carries additional risks
* ESO allows us a certain level of independence from the Secrets Management System
* HashiCorp Vault is currently the most popular Secret Management Systems
* Advised by our security team
* CyberArk leaders (masters) cannot be hosted on OpenShift
* Secret injection is incompatible with Pods owned by Operators


[#approaches]
== Approaches

There are two fundamental approaches to manage secret material within a GitOps context:

* **Encrypted Secrets** stored inside Git repositories
* **References to Secrets** stored inside Git repositories and the actual secret is stored somewhere else

[#encryptedSecrets]
=== Encrypted Secrets

* Secrets are stored in an encrypted form inside the Git repository
* Automation and helper tools do the decryption to create Kubernetes secrets from them
* A number of projects exist that implement this approach:
** link:https://github.com/bitnami-labs/sealed-secrets[Sealed Secrets]
** link:https://github.com/mozilla/sops[Mozilla's Secret OPerationS (SOPS)]
** Other smaller ones (link:https://github.com/kapicorp/tesoro[Tesoro], link:https://github.com/Soluto/kamus[Kamus])

These projects provide an easy way to encrypt and decrypt the secrets, and make sure that only authorized users can access them.
The encryption method and key management should be considered carefully as they are important to ensure the security of the secrets.

IMPORTANT: You should evaluate and choose the right project that fits your specific use case and requirements.

[#secretReferences]
=== References to Secrets

This approach requires two main parts:
* A Secret Management System (link:https://www.vaultproject.io/[HashiCorp Vault], link:https://www.conjur.org/[Conjur], link:https://lyft.github.io/confidant/[Confidant])
* A controller that retrieves the secret from the Secret Management System and translates it to a Kubernetes Secret or injects it into the Pod directly (link:https://external-secrets.io/v0.7.0/[External Secrets Operator], link:https://github.com/kubernetes-sigs/secrets-store-csi-driver[Kubernetes Secrets Store CSI Driver], link:https://developer.hashicorp.com/vault/docs/platform/k8s/injector[Vault Agent Injector])
* Secrets are uploaded/created directly into the Secret Management System
* References to secrets are stored in Git
* A controller reads the references to secrets and translates them into Kubernetes secrets

IMPORTANT: HashiCorp Vault runs only on the HUB

[#vault]
* The Vault has multiple kubernetes backends configured by the unsealVault Cron Job (one for each cluster)
* There is one ESO instance for each cluster (ESO is currently unsupported)
* Each ESO instance logs into the HashiCorp Vault using a specific login path that was configured with the credentials of that instance
* Vault will authenticate the requests by talking to the API endpoints of the clusters where the login request originated from

[#eso]

* At a high-level the External Secrets Operator reads secrets from the Vault and converts them into Kubernetes Secrets
* To populate the Vault with our secret material we use `make load-secrets`
* Normally a user would create ~/values-secret-<patternname>.yaml following the contents of values-secret.yaml.template in the pattern's git repository
* Running `./pattern.sh make load-secrets` will populate the Vault with our secrets

NOTE: Provisioning the vault can be complex, so it is taken care of by the framework

* Vault unsealing and configuration happens in the **imperative** namespace with a CronJob

NOTE: The Unseal vault cron job is idempotent and runs every five minutes on the hub only

image::unseal-vault-cronjob.png[]

* The **vaultkeys** secret containing the keys to unseal the vault and the vault root token that can be used to login to the vault's UI is created

IMPORTANT: It is **strongly** recommended that the **vaultkeys** secret in the **imperative** namespace is exported to a safe space and removed. It contains the keys needed to unseal the vault whenever it gets restarted

[#vaultconcepts]
=== Vault Concepts

* Secrets are stored in **Vault** in specific **paths**
* **secret/global/config-demo** is the **path**
* Each **path** can have multiple attributes
* **secret** is one attribute at this path
* In this case secret is autogenerated by default inside the vault directly

[.INFORMATION]
====
By default there are three paths that can be used

* **secret/global/*** accessible by the hub and all managed clusters
* **secret/hub/*** accessible only by the hub cluster
* **secret/<fqdn of managed cluster>/*** accessible only by the fqdn-defined cluster
====

TIP: You can think of a path as being a single dictionary with multiple keys. An external secret looks for a path and then has a templating system to extract keys and put them in kubernetes secrets.

[source,yaml]
----
secrets:
  - name: config-demo
    vaultPrefixes:
    - global
    fields:
    - name: secret
      onMissingValue: generate
      vaultPolicy: validatedPatternDefaultPolicy
----

From the vault interface perspective:

image::global-configdemo-vault.png[]

[#esoconcepts]
=== ESO Concepts

ESO will read secrets from vault paths (e.g. "`secret/data/global/config-demo`" is a path)

This ExternalSecret object will read all the attributes of the vault key `secret/global/config-demo` and will create a secret called `config-demo-secret` with one entry per attribute.

[source,yaml]
----
apiVersion: "external-secrets.io/v1beta1"
kind: ExternalSecret
metadata:
  name: config-demo-secret
  namespace: config-demo
spec:
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: config-demo-secret
    template:
      type: Opaque
  dataFrom:
  - extract:
      key: secret/data/global/config-demo
----

NOTE: How secret material ends up in the vault at a specific path is explained in detail in an upcoming section
