This tutorial demonstrates the K/V Secrets Engine version 2 with secret versioning.
The KV secrets engine v1 does not provide away to version or roll back secrets. This made it difficult to recover fromunintentional data loss or overwrite when more than one user is writing at thesame path.
Solution
Run the version 2 of KV secrets engine which can retain a configurablenumber of secret versions. This enables older versions' data to be retrievablein case of unwanted deletion or updates of the data. In addition, itsCheck-and-Set operations can be used to protect the data from being overwrittenunintentionally.
What this tutorial covers
This tutorial will walk you through the basic features of the KV v2 secretsengine:
- Step 1: Check the KV secrets engine version
- Step 2: Write secrets
- Patch the existing data
- Add custom metadata
- Step 3: Retrieve a specific version of secret
- Step 4: Specify the number of versions to keep
- Step 5: Delete versions of secret
- Step 6: Permanently delete data
- Step 7: Configure automatic data deletion
- Step 8: Check-and-Set operations
To perform the tasks described in this tutorial, you need to have a Vaultenvironment. Refer to the GettingStarted tutorial to install Vault.
Launch Terminal
This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.
Policy requirements
Note For the purpose of this tutorial, you can use root
token to workwith Vault. However, it is recommended that root tokens are only used forinitial setup or in emergencies. As a best practice, use tokens withappropriate set of policies based on your role in the organization.
To perform all tasks demonstrated in this tutorial, your policy must include thefollowing permissions:
# Write and manage secrets in key-value secrets enginepath "secret*" { capabilities = [ "create", "read", "update", "delete", "list", "patch" ]}# To enable secrets enginespath "sys/mounts/*" { capabilities = [ "create", "read", "update", "delete" ]}
Vault version The -output-policy
flag requires Vault 1.11.0 or later.
To perform all tasks demonstrated in this tutorial, you are going to need to runa number of vault
CLI commands. Adding the -output-policy
flag does notexecute the command, but instead prints the ACL policy needed to successfullyexecute the command.
Check the policy needed for kv put
:
$ vault kv put -output-policy secret/customer/acme customer_name="ACME Inc." \ contact_email="[email protected]"
Output:
path "secret/data/customer/acme" { capabilities = ["create", "update"]}
Now, check the ACL policy needed for kv get
:
$ vault kv get -output-policy secret/customer/acmepath "secret/data/customer/acme" { capabilities = ["read"]}
If you are not familiar with policies, complete thepolicies tutorial.
Lab setup
Note If you do not have access to an HCP Vault Dedicated cluster, visit the Create a Vault Cluster on HCP tutorial.
Launch the HCP Portal and login.
Click Vault in the left navigation pane.
In the Vault clusters pane, click vault-cluster.
Under Cluster URLs, click Public Cluster URL.
In a terminal, set the
VAULT_ADDR
environment variable to the copiedaddress.$ export VAULT_ADDR=<Public_Cluster_URL>
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
Return to the terminal and set the
VAULT_TOKEN
environment variable.$ export VAULT_TOKEN=<token>
Set the
VAULT_NAMESPACE
environment variable toadmin
.$ export VAULT_NAMESPACE=admin
The
admin
namespace is the top-level namespace automatically created by HCPVault. All CLI operations default to use the namespace defined in thisenvironment variable.Type
vault status
to verify your connectivity to the Vault cluster.$ vault statusKey Value--- -----Recovery Seal Type shamirInitialized trueSealed falseTotal Recovery Shares 1Threshold 1Version 1.9.2+entStorage Type raft...snipped...
The Vault Dedicated server is ready.
(Persona: admin)
The Vault server started in devmode, automatically enables v2
of the KV secrets engine at the secret/
path. Verify that KV secrets engineis enabled and is set to version 2.
Display all the enabled secrets engine.
$ vault secrets list -detailedPath Type Accessor ... Options Description---- ---- -------- ------- -----------cubbyhole/ cubbyhole cubbyhole_9d52aeac ... map[] per-token private secret storageidentity/ identity identity_acea5ba9 ... map[] identity storesecret/ kv kv_2226b7d3 ... map[version:2] key/value secret storage...
The results display a table of all enabled secrets engines. One entry in thetable has the Path of secret/
with the Type of kv
. The Optionscolumn displays the version number.
Enable KV v2 at secret/
.
$ vault secrets enable -path=secret kv-v2
If the KV version is version:1
, upgrade it to version:2
.
$ vault kv enable-versioning secret/
Display all the enabled secrets engine.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/sys/mounts | jq
Note This example uses jq toprocess the JSON output for readability.
Example output:
...snip... "secret/": { "accessor": "kv_f42ae9d1", "config": { "default_lease_ttl": 0, "force_no_cache": false, "max_lease_ttl": 0 }, "description": "key/value secret storage", "external_entropy_access": false, "local": false, "options": { "version": "2" }, "seal_wrap": false, "type": "kv", "uuid": "b3ffe87c-4479-552c-7338-946deda76b39" },...snip...
The results display all the enabled secrets engines. One entry in the resultshas the path of secret/
with the type
of kv
. The options
displays theversion number.
Enable KV v2 at secret/
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{ "type": "kv-v2" }' \ $VAULT_ADDR/v1/sys/mounts/secret
If the KV version is 1
, upgrade it to 2
.
Create an API request payload specifying the version.
$ tee payload.json <<EOF{ "options": { "version": "2" }}EOF
Upgrade the secrets engine at path secret
to version 2.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload.json \ $VAULT_ADDR/v1/sys/mounts/secret/tune
Open a web browser and launch the Vault UI (e.g. http://localhost:8200/ui
) andthen login.
Note If secret/
does NOT indicate as v2 you can upgrade it from v1
to v2
.
Click the Vault CLI shell icon (>_
) to open a command shell. Execute thefollowing command to upgrade it to v2
:
$ vault write sys/mounts/secret/tune version=2
Note If KV secrets engine is not enabled, enable KV v2 secrets engineat secret/
path.
Click Enable new engine. Select KV from the list, and then clickNext.
Enter
secret
in the Path field.Expand Method Options and make sure the Version is set as 2.
Click Enable Engine to complete.
Step 2: Write secrets
To understand how the versioning works, let's write some test data.
Create a secret at the path secret/customer/acme
with a name
andcontact_email
.
$ vault kv put secret/customer/acme customer_name="ACME Inc." \ contact_email="[email protected]"
New in Vault 1.11.0: The following command is equivalent to the abovecommand.
$ vault kv put -mount=secret customer/acme customer_name="ACME Inc." \ contact_email="[email protected]"
Example output:
====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T13:41:45.673767Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 1
The secret stores metadata along with the secret data. The version
is an auto-incrementing number that starts at 1
.
Create secret at the same path secret/customer/acme
but with differentsecret data.
$ vault kv put secret/customer/acme customer_name="ACME Inc." \ contact_email="[email protected]"
Example output:
====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T13:41:45.673767Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 2
The secret is fully replaced and the version is incremented to 2
.
Get the secret defined at the path secret/customer/acme
.
$ vault kv get secret/customer/acme
New in Vault 1.11.0: The following command is equivalent to the abovecommand.
$ vault kv get -mount=secret customer/acme
Example output:
====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T12:59:47.15049Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 2======== Data ========Key Value--- -----contact_email [email protected]name ACME Inc.
The output displays version 2 of the secret. Creating a secret at the same pathreplaces the existing data; fields are not merged together. The secret datastored in earlier versions is still accessible but it is no longer returned bydefault.
Patch the existing data
While vault kv put
fully replaces the current version of the secret;therefore, you need to send the entire set of data including the values thatremain the same. To partially update the current version of the secret, you canuse vault kv patch
command instead.
Use case: Think of an application that does not have read permission, butcaptures partial data updates. The vault kv patch
command allows theapplication to send only the changing values to Vault.
ACL Policy KV v2 secrets engine honors the distinction between thecreate
and update
capabilities inside ACL policies. The patch
capabilityis also supported which is used to represent partial updates whereas theupdate
capability represents full overwrites. To permit partial updates, besure to add patch
capability in the ACL policy.
Update the contact_email
value to [email protected]
(current value is[email protected]
).
$ vault kv patch secret/customer/acme contact_email="[email protected]"
Example output:
====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T13:40:17.992243Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 6
The patch command creates a new version of the secret which merges the fieldswithin the secret data.
Read the secret at secret/customer/acme
to verify the change.
$ vault kv get secret/customer/acme====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T13:41:45.673767Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 3======== Data ========Key Value--- -----contact_email [email protected]customer_name ACME Inc.
The contact_email
value is updated to [email protected]
, and the version
isnow 3.
Add custom metadata
You saw that the secret's key (in this example, secret/customer/acme
) hasmetadata associated with it. An organization may wish to add custom metadatadescribing further details (technical contact, mission criticality, etc.) aboutthe secret.
When you are running Vault 1.9.0 or later, you can add custom metadata usingcustom_metadata
parameter. This feature stores a map of arbitrarystring-to-string valued user-provided metadata meant to describe the secret.
$ vault kv metadata put -custom-metadata=Membership="Platinum" secret/customer/acmeSuccess! Data written to: secret/metadata/customer/acme
The -custom-metadata
flag can be repeated to add multiple key-value pairs.
$ vault kv metadata put -custom-metadata=Membership="Platinum" \ -custom-metadata=Region="US West" secret/customer/acme
Output:
Success! Data written to: secret/metadata/customer/acme
Now, when you read the secret, the returned secret metadata displays the custommetadata.
$ vault kv get secret/customer/acme
Example output:
====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T13:41:45.673767Zcustom_metadata map[Membership:Platinum Region:US West]deletion_time n/adestroyed falseversion 3======== Data ========Key Value--- -----contact_email [email protected]customer_name ACME Inc.
Create an API request payload containing some test data.
$ tee payload.json <<EOF{ "data": { "name": "ACME Inc.", "contact_email": "[email protected]" }}EOF
Write some data at secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload.json \ $VAULT_ADDR/v1/secret/data/customer/acme | jq
Example output:
{ "request_id": "e8021c29-394e-faee-ecab-9a2d55a6f622", "lease_id": "", "renewable": false, "lease_duration": 0, "data": { "created_time": "2021-10-29T02:08:37.389337Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 1 }, "wrap_info": null, "warnings": null, "auth": null}
Notice that the endpoint for KV v2 is /secret/data/<path>
; therefore towrite secrets at secret/customer/acme
, the API endpoint becomes/secret/data/customer/acme
.
Create an API request payload containing the data to update the current data.
$ tee payload_2.json <<EOF{ "data": { "customer_name": "ACME Inc.", "contact_email": "[email protected]" }}EOF
Update the secret to create another version.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload_2.json \ $VAULT_ADDR/v1/secret/data/customer/acme | jq
Example output:
{ "request_id": "8fcdec0f-99bb-a62a-20be-e3239e8a2d7a", "lease_id": "", "renewable": false, "lease_duration": 0, "data": { "created_time": "2021-10-29T02:09:32.112647Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 2 }, "wrap_info": null, "warnings": null, "auth": null}
Now you have two versions of the secret/customer/acme
data.
Read back the secret.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/data/customer/acme | jq -r ".data"
Example output:
{ "data": { "contact_email": "[email protected]", "customer_name": "ACME Inc." }, "metadata": { "created_time": "2021-10-29T02:09:32.112647Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 2 }}
Patch the existing data
While POST
fully replaces the current version of the secret; therefore, youneed to send the entire set of data including the values that remain the same.When you are running Vault 1.9.0 or later, you can use PATCH
to partiallyupdate the current version of the secret.
Use case: Use case: Think of an application that does not have readpermission, but captures partial data updates. The PATCH
operation allows theapplication to send only the changing values to Vault.
ACL Policy KV v2 secrets engine honors the distinction between thecreate
and update
capabilities inside ACL policies. The patch
capabilityis also supported which is used to represent partial updates whereas theupdate
capability represents full overwrites. To permit partial updates, besure to add patch
capability in the ACL policy.
Update the contact_email
value to [email protected]
(current value is[email protected]
).
Create an API request payload containing the data to update the current data.
$ tee payload_patch.json <<EOF{ "data": { "contact_email": "[email protected]" }}EOF
Update the secret to create another version.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --header "Content-Type: application/merge-patch+json" \ --request PATCH \ --data @payload_patch.json \ $VAULT_ADDR/v1/secret/data/customer/acme | jq
This uses JSON mergepatchand must be specified using a Content-Type
header value ofapplication/merge-patch+json
.
Example output:
{ "request_id": "deadc6ad-473d-2433-ae2c-5c4a2fecbbc6", "lease_id": "", "renewable": false, "lease_duration": 0, "data": { "created_time": "2021-10-29T02:11:45.005824Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 3 }, "wrap_info": null, "warnings": null, "auth": null}
Read back the secret.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/data/customer/acme | jq -r ".data"
Example output:
{ "data": { "contact_email": "[email protected]", "customer_name": "ACME Inc." }, "metadata": { "created_time": "2021-10-29T02:11:45.005824Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 3 }}
The contact_email
value is updated to [email protected]
, adn the version
isnow 3.
Add custom metadata
Note This section describes new feature introduced in Vault 1.9.0.
You saw that the secret's key (in this example, secret/customer/acme
) hasmetadata associated with it. An organization may wish to add custom metadatadescribing further details (technical contact, mission criticality, etc.) aboutthe secret.
When you are running Vault 1.9.0 or later, you can add custom metadata usingcustom_metadata
parameter. This feature stores a map of arbitrarystring-to-string valued user-provided metadata meant to describe the secret.
Create an API request payload containing the custom metadata.
$ tee payload_metadata.json <<EOF{ "custom_metadata": { "Membership": "Platinum", "Region": "US West" }}EOF
Update the secret key metadata.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload_metadata.json \ $VAULT_ADDR/v1/secret/metadata/customer/acme
Read back the secret.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/data/customer/acme | jq -r ".data"
Example output:
{ "data": { "contact_email": "[email protected]", "customer_name": "ACME Inc." }, "metadata": { "created_time": "2021-10-29T02:11:45.005824Z", "custom_metadata": { "Membership": "Platinum", "Region": "US West" }, "deletion_time": "", "destroyed": false, "version": 3 }}
The returned secret metadata displays the custom metadata.
In the Web UI, select
secret/
and then click Create secret. Entercustomer/acme
in the Path for this secret field.Under Secret data, enter
name
in the key field, andACME Inc.
in itsvalue field. You can click on the sensitive information toggle to show theentered secret values.Click Add.
Enter
contact_email
in the second key field, and[email protected]
as itsvalue.Click Save.
To update the existing secret, select Create new version.s
Change the
contact_email
value (e.g.[email protected]
) and then clickSave.
Add custom metadata
Note This section describes new feature introduced in Vault 1.9.0.
You saw that the secret's key (in this example, secret/customer/acme
) hasmetadata associated with it. An organization may wish to add custom metadatadescribing further details (technical contact, mission criticality, etc.) aboutthe secret.
Select the Metadata tab, and then click Add metadata.
Under Custom metadata, enter
Membership
in the key field, andPlatinum
inits value field.Click Add.
Enter
Region
in the second key field, andUS West
in its value field.Click Save. Now, metadata is set on the
secret/customer/acme
key.
You may run into a situation where you need to view the secret before an update.
Get version 1 of the secret defined at the path secret/customer/acme
.
$ vault kv get -version=13 secret/customer/acme====== Secret Path ======secret/data/customer/acme======= Metadata =======Key Value--- -----created_time 2022-06-13T15:09:01.710331Zcustom_metadata map[Membership:Platinum Region:US West]deletion_time n/adestroyed falseversion 13======== Data ========Key Value--- -----contact_email [email protected]customer_name ACME Inc.
Get the metadata
of the secret defined at the path secret/customer/acme
.
$ vault kv metadata get secret/customer/acme======= Metadata Path =======secret/metadata/customer/acme========== Metadata ==========Key Value--- -----cas_required falsecreated_time 2021-10-31T00:08:46.869191Zcurrent_version 3custom_metadata map[Membership:Platinum Region:US West]delete_version_after 0smax_versions 0oldest_version 0updated_time 2021-10-31T00:10:14.147236Z====== Version 1 ======Key Value--- -----created_time 2021-10-31T00:08:46.869191Zdeletion_time n/adestroyed false====== Version 2 ======Key Value--- -----created_time 2021-10-31T00:09:32.659503Zdeletion_time n/adestroyed false====== Version 3 ======Key Value--- -----created_time 2021-10-31T00:10:14.147236Zdeletion_time n/adestroyed false
Get version 1 of the secret defined at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/data/customer/acme\?version=1 | jq -r ".data"
Example output:
{ "data": { "contact_email": "[email protected]", "name": "ACME Inc." }, "metadata": { "created_time": "2021-10-29T02:08:37.389337Z", "custom_metadata": { "Membership": "Platinum", "Region": "US West" }, "deletion_time": "", "destroyed": false, "version": 1 }}
Get the metadata of the secret defined at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/metadata/customer/acme | jq -r ".data"
Example output:
{ "cas_required": false, "created_time": "2021-10-29T02:08:37.389337Z", "current_version": 3, "custom_metadata": { "Membership": "Platinum", "Region": "US West" }, "delete_version_after": "0s", "max_versions": 0, "oldest_version": 0, "updated_time": "2021-10-29T02:11:45.005824Z", "versions": { "1": { "created_time": "2021-10-29T02:08:37.389337Z", "deletion_time": "", "destroyed": false }, "2": { "created_time": "2021-10-29T02:09:32.112647Z", "deletion_time": "", "destroyed": false }, "3": { "created_time": "2021-10-29T02:11:45.005824Z", "deletion_time": "", "destroyed": false } }}
Click the Secret tab.
Select the down-arrow next to Version 2 and then Version 1
The version changes from
Version 2
toVersion 1
. To view the raw values,click on the sensitive information toggle.To create a new version of the secrets based on the Version 1 data, Createnew version +.
This can help revert an older version of the secrets.
Step 4: Specify the number of versions to keep
By default, the kv-v2
secrets engine keeps up to 10 versions. Let's limit themaximum number of versions to keep to be 4.
Configure the secrets engine at path secret/
to limit all secrets to amaximum of 4
versions.
$ vault write secret/config max_versions=4Success! Data written to: secret/config
Every secret stored for this engine are set to a maximum of 4
versions.
Display the secrets engine configuration settings.
$ vault read secret/configKey Value--- -----cas_required falsedelete_version_after 0smax_versions 4
Configure the secret at path secret/customer/acme
to limit secrets toa maximum of 4
versions.
$ vault kv metadata put -max-versions=4 secret/customer/acmeSuccess! Data written to: secret/metadata/customer/acme
The secret can also define the maximum number of versions.
Create four more secrets at the path secret/customer/acme
.
$ vault kv put secret/customer/acme name="ACME Inc." \ contact_email="[email protected]"
Get the metadata
of the secret defined at the path secret/customer/acme
.
$ vault kv metadata get secret/customer/acme
Example output:
======= Metadata Path =======secret/metadata/customer/acme========== Metadata ==========Key Value--- -----cas_required falsecreated_time 2021-10-31T00:08:46.869191Zcurrent_version 7custom_metadata map[Membership:Platinum Region:US West]delete_version_after 0smax_versions 4oldest_version 4updated_time 2021-10-31T00:15:04.591209Z====== Version 4 ======Key Value--- -----created_time 2021-10-31T00:14:59.830407Zdeletion_time n/adestroyed false====== Version 5 ======Key Value--- -----created_time 2021-10-31T00:15:01.892226Zdeletion_time n/adestroyed false====== Version 6 ======Key Value--- -----created_time 2021-10-31T00:15:03.418051Zdeletion_time n/adestroyed false====== Version 7 ======Key Value--- -----created_time 2021-10-31T00:15:04.591209Zdeletion_time n/adestroyed false
The metadata displays the current_version
and the history of versions stored.Secrets stored at this path are limited to 4 versions. Version 1 and 2 aredeleted.
Verify that version 1 of the secret defined at the path secret/customer/acme
are deleted.
$ vault kv get -version=1 secret/customer/acmeNo value found at secret/data/customer/acme
Create an API request payload specifying the max_versions
to 4
.
$ tee payload-config.json<<EOF{ "max_versions": 4, "cas_required": false}EOF
Configure the secrets engine at path secret/
to limit all secrets to amaximum of 4
versions.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload-config.json \ $VAULT_ADDR/v1/secret/config
Every secret stored for this engine are set to a maximum of 4
versions.
Display the secrets engine configuration settings.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/config | jq -r ".data"
The output should as follow:
{ "cas_required": false, "delete_version_after": "0s", "max_versions": 4}
Configure the secret at path secret/customer/acme
to limit secrets toa maximum of 4
versions.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload.json \ $VAULT_ADDR/v1/secret/metadata/customer/acme
The secret can also define the maximum number of versions.
Create four more secrets at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload.json \ $VAULT_ADDR/v1/secret/data/customer/acme
Get the metadata
of the secret defined at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/metadata/customer/acme | jq -r ".data"
Example output:
{ "cas_required": false, "created_time": "2021-10-29T02:08:37.389337Z", "current_version": 7, "custom_metadata": { "Membership": "Platinum", "Region": "US West" }, "delete_version_after": "0s", "max_versions": 0, "oldest_version": 4, "updated_time": "2021-10-29T02:57:54.601033Z", "versions": { "4": { "created_time": "2021-10-29T02:55:31.078902Z", "deletion_time": "", "destroyed": false }, "5": { "created_time": "2021-10-29T02:57:13.519347Z", "deletion_time": "", "destroyed": false }, "6": { "created_time": "2021-10-29T02:57:21.096945Z", "deletion_time": "", "destroyed": false }, "7": { "created_time": "2021-10-29T02:57:54.601033Z", "deletion_time": "", "destroyed": false } }}
The metadata displays the current_version
and the history of versions stored.Secrets stored at this path are limited to 4 versions. Version 1, 2, and 3 aredeleted.
Verify that version 1 of the secret defined at the path secret/customer/acme
are deleted.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/data/customer/acme\?version=1 | jq
The output should look as follow:
{ "errors": []}
Click the Vault CLI shell icon (
>_
) to open a command shell. Execute thefollowing command to tune the maximum number of versions to keep:$ vault write secret/config max_versions=4
Click the icon (
>_
) again to hide the shell.Overwrite the data a few more times to see what happens to its versions.
In this example, the current version is 6. Notice that version 1 and 2 do notshow up in the metadata. Because the kv secrets engine is configured to keep only4 versions, the oldest two versions are permanently deleted and you won't beable to read them.
Delete version 4 and 5 of the secrets at path secret/customer/acme
.
$ vault kv delete -versions="4,5" secret/customer/acmeSuccess! Data deleted (if it existed) at: secret/customer/acme
Get the metadata of the secret defined at the path secret/customer/acme
.
$ vault kv metadata get secret/customer/acme##...snip...====== Version 4 ======Key Value--- -----created_time 2021-10-31T00:14:59.830407Zdeletion_time 2021-10-31T00:16:25.860618Zdestroyed false====== Version 5 ======Key Value--- -----created_time 2021-10-31T00:15:01.892226Zdeletion_time 2021-10-31T00:16:25.860619Zdestroyed false##...snip...
The metadata on versions 4 and 5 reports its deletion timestamp(deletion_time
); however, the destroyed
parameter is set to false
.
Undelete version 5 of the secrets at path secret/customer/acme
.
$ vault kv undelete -versions=5 secret/customer/acmeSuccess! Data written to: secret/undelete/customer/acme
Delete version 4 and 5 of the secrets at path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{ "versions":[4,5] }' \ $VAULT_ADDR/v1/secret/delete/customer/acme
Get the metadata of the secret defined at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/metadata/customer/acme | jq -r ".data.versions"
Example output:
{ "4": { "created_time": "2021-10-29T02:55:31.078902Z", "deletion_time": "2021-10-29T03:05:26.946674Z", "destroyed": false }, "5": { "created_time": "2021-10-29T02:57:13.519347Z", "deletion_time": "2021-10-29T03:05:26.946675Z", "destroyed": false }, "6": { "created_time": "2021-10-29T02:57:21.096945Z", "deletion_time": "", "destroyed": false }, "7": { "created_time": "2021-10-29T02:57:54.601033Z", "deletion_time": "", "destroyed": false }}
The metadata on versions 4 and 5 reports its deletion timestamp(deletion_time
); however, the destroyed
parameter is set to false
.
You can undelete version 5 of the secrets at path secret/customer/acme
if itwas deleted accidentally.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{ "versions":[5] }' \ $VAULT_ADDR/v1/secret/undelete/customer/acme
Delete versions 4 and 5.
At the
secret/customer/acme
path, select the down-arrow next to Version 6 and then select Version 5, and then click Delete.At the confirmation dialog, the Delete this version radio button, and then click Delete to proceed.
If version 5 was deleted by mistake and you wish to recover, selectUndelete from then menu.
Delete Version 4. Now, Version 4 should be marked as deleted.
Select Version 4 and then View version history. Here you can see the state of all versions.
Step 6: Permanently delete data
Destroy version 4 of the secrets at path secret/customer/acme
.
$ vault kv destroy -versions=4 secret/customer/acmeSuccess! Data written to: secret/destroy/customer/acme
Get the metadata of the secret defined at the path secret/customer/acme
.
$ vault kv metadata get secret/customer/acme##...snip...====== Version 4 ======Key Value--- -----created_time 2021-10-31T00:14:59.830407Zdeletion_time 2021-10-31T00:16:25.860618Zdestroyed true##...snip...
The metadata displays that Version 4 is destroyed.
Delete all versions of the secret at the path secret/customer/acme
.
$ vault kv metadata delete secret/customer/acmeSuccess! Data deleted (if it existed) at: secret/metadata/customer/acme
Destroy version 4 of the secrets at path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{ "versions":[4] }' \ $VAULT_ADDR/v1/secret/destroy/customer/acme
Get the metadata of the secret defined at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/metadata/customer/acme | jq -r ".data.versions"
Example output:
{ "4": { "created_time": "2021-10-29T02:55:31.078902Z", "deletion_time": "2021-10-29T03:05:26.946674Z", "destroyed": true }, "5": { "created_time": "2021-10-29T02:57:13.519347Z", "deletion_time": "", "destroyed": false }, "6": { "created_time": "2021-10-29T02:57:21.096945Z", "deletion_time": "", "destroyed": false }, "7": { "created_time": "2021-10-29T02:57:54.601033Z", "deletion_time": "", "destroyed": false }}
The metadata indicates that Version 4 is destroyed.
Delete all versions of the secret at the path secret/customer/acme
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request DELETE \ $VAULT_ADDR/v1/secret/metadata/customer/acme
To permanently delete a version of secret, simply select Permanently destroyversion from its menu.
Select Version 4 from the menu and switch to Version 3.
Select Delete, choose Destroy this version and then click Delete.
Select Version 3 from the menu and switch to View version history.
You can see that Version 3 is Destroyed and Version 4 is Deleted.
As of Vault 1.2, you can configure the length of time before a version getsdeleted. For example, if your organization requires data to be deleted after 10days from its creation, you can configure the K/V v2 secrets engine to do so bysetting the delete_version_after
parameter.
For demonstration, configure the secrets at path secret/test
to deleteversions after 40
seconds.
$ vault kv metadata put -delete-version-after=40s secret/test 10:52:05Success! Data written to: secret/metadata/test
Create a secret at the path secret/test
.
$ vault kv put secret/test message="data1"
Again, create a secret at the path secret/test
.
$ vault kv put secret/test message="data2"
Again, create a secret at the path secret/test
.
$ vault kv put secret/test message="data3"
Note You can use upper-arrow key to recover the previously executedcommand.
Get the metadata of the secret defined at the path secret/test
.
$ vault kv metadata get secret/test
Example output:
=== Metadata Path ===secret/metadata/test========== Metadata ==========Key Value--- -----cas_required falsecreated_time 2021-05-28T19:03:03.618924Zcurrent_version 3delete_version_after 40smax_versions 0oldest_version 0updated_time 2021-05-28T19:03:20.857496Z====== Version 1 ======Key Value--- -----created_time 2021-05-28T19:03:09.818797Zdeletion_time 2021-05-28T19:03:49.818797Zdestroyed false====== Version 2 ======Key Value--- -----created_time 2021-05-28T19:03:16.991649Zdeletion_time 2021-05-28T19:03:56.991649Zdestroyed false====== Version 3 ======Key Value--- -----created_time 2021-05-28T19:03:20.857496Zdeletion_time 2021-05-28T19:04:00.857496Zdestroyed false
The metadata displays a deletion_time
set on each version. After 40 seconds,the data gets deleted automatically. The data has not been destroyed.
Get version 1 of the secret defined at the path secret/test
.
$ vault kv get -version=1 secret/test== Secret Path ==secret/data/test====== Metadata ======Key Value--- -----created_time 2021-05-28T19:03:09.818797Zdeletion_time 2021-05-28T19:03:49.818797Zdestroyed falseversion 1===== Data =====Key Value--- -----message data1
For demonstration, configure the secrets at path secret/test
to deleteversions after 40
seconds.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{ "delete_version_after": "40s" }' \ $VAULT_ADDR/v1/secret/metadata/test
Create a secret at the path secret/test
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{"data": {"message": "data1"}}' \ $VAULT_ADDR/v1/secret/data/test | jq -r ".data"
Example output:
{ "created_time": "2021-10-29T03:15:39.375151Z", "custom_metadata": null, "deletion_time": "2021-10-29T03:16:19.375151Z", "destroyed": false, "version": 1}
Again, create a secret at the path secret/test
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{"data": {"message": "data2"}}' \ $VAULT_ADDR/v1/secret/data/test
Again, create a secret at the path secret/test
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{"data": {"message": "data3"}}' \ $VAULT_ADDR/v1/secret/data/test
Get the metadata of the secret defined at the path secret/test
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/metadata/test | jq -r ".data"
Example output:
{ "cas_required": false, "created_time": "2021-10-29T03:15:31.743167Z", "current_version": 3, "custom_metadata": null, "delete_version_after": "40s", "max_versions": 0, "oldest_version": 0, "updated_time": "2021-10-29T03:16:11.848245Z", "versions": { "1": { "created_time": "2021-10-29T03:15:39.375151Z", "deletion_time": "2021-10-29T03:16:19.375151Z", "destroyed": false }, "2": { "created_time": "2021-10-29T03:16:03.366448Z", "deletion_time": "2021-10-29T03:16:43.366448Z", "destroyed": false }, "3": { "created_time": "2021-10-29T03:16:11.848245Z", "deletion_time": "2021-10-29T03:16:51.848245Z", "destroyed": false } }}
The metadata displays a deletion_time
set on each version. After 40 seconds,the data gets deleted automatically. The data has not been destroyed.
Get version 1 of the secret defined at the path `secret/test.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ $VAULT_ADDR/v1/secret/data/test\?version=1 | jq -r ".data"
Example output:
{ "data": null, "metadata": { "created_time": "2021-10-29T03:15:39.375151Z", "custom_metadata": null, "deletion_time": "2021-10-29T03:16:19.375151Z", "destroyed": false, "version": 1 }}
Note This section describes the new UI feature introduced in Vault1.9.0. If you are running Vault 1.2 or later, but earlier than 1.9, use theVault CLI or API to configure automatic data deletion.
For demonstration, configure the secrets at path secret/test
to deleteversions after 40
seconds.
In the Web UI, select
secret/
and then click Create secret.Enter
test
in the Path for this secret field.Under Secret data, enter
message
in the key field, anddata1
in itsvalue field.Expand Show secret metadata.
Slide the Automate secret deletion toggle to enable it, and enter
40
toset the secrets at this path to be deleted after 40 seconds.Click Save.
Click Create new version +, and then click Save to create a newversion of the secret.
Repeat the step a couple of more times to create multiple versions of thesecret.
Select the Metadata tab, and you should see that the Delete versionafter parameter is set to
40s
.Wait for 40 seconds and select the Secret tab.
Because all created secrets were deleted automatically after 40 seconds, theUI shows no key values.
Step 8: Check-and-Set operations
The v2 of KV secrets engine supports a Check-And-Set operation to preventunintentional secret overwrite. When you pass the cas
flag to Vault, it firstchecks if the key already exists.
Display the secrets engine configuration settings.
$ vault read secret/configKey Value--- -----cas_required falsedelete_version_after 0smax_versions 4
The cas_required
setting is false
. The KV secrets engine defaults to disablethe Check-And-Set operation.
Configure the secrets engine at path secret/
to enable Check-And-Set.
$ vault write secret/config cas_required=true
Configure the secret at path secret/partner
to enable Check-And-Set.
$ vault kv metadata put -cas-required=true secret/partner
Once check-and-set is enabled, every write operation requires the cas
parameter with the current version of the secret. Set cas
to 0
when a secretat that path does not already exist.
Create a new secret at the path secret/partner
.
$ vault kv put -cas=0 secret/partner name="Example Co." partner_id="123456789"Key Value--- -----created_time 2021-05-28T19:05:50.165496Zdeletion_time n/adestroyed falseversion 1
Overwrite the secret at the path secret/partner
.
$ vault kv put -cas=1 secret/partner name="Example Co." \ partner_id="ABCDEFGHIJKLMN"
Example output:
Key Value--- -----created_time 2021-05-28T19:06:57.350072Zdeletion_time n/adestroyed falseversion 2
Create an API request payload that sets cas_required
to true
.
$ tee payload-cas.json<<EOF{ "max_versions": 10, "cas_required": true}EOF
Configure the secrets engine at path secret/
to enable Check-And-Set.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload-cas.json \ $VAULT_ADDR/v1/secret/config
Configure the secret at path secret/partner
to enable Check-And-Set.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data '{"cas_required": true}' \ $VAULT_ADDR/v1/secret/metadata/partner
Once check-and-set is enabled, every write operation requires the cas
parameter with the current version of the secret. Set cas
to 0
when a secretat that path does not already exist.
Create an API request payload with secret data and the cas
value to 0
.
$ tee payload_3.json <<EOF{ "options": { "cas": 0 }, "data": { "name": "Example Co.", "partner_id": "123456789" }}EOF
Create a new secret at the path secret/partner
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload_3.json \ $VAULT_ADDR/v1/secret/data/partner | jq -r ".data"
Example output:
{ "created_time": "2021-10-31T01:08:29.152955Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 1}
Create an API request payload with updated secret data and the cas
value to1
.
$ tee payload_4.json <<EOF{ "options": { "cas": 1 }, "data": { "name": "Example Co.", "partner_id": "ABCDEFGHIJKLMN" }}EOF
Overwrite the secret at the path secret/partner
.
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \ --header "X-Vault-Namespace: $VAULT_NAMESPACE" \ --request POST \ --data @payload_4.json \ $VAULT_ADDR/v1/secret/data/partner | jq -r ".data"
Example output:
{ "created_time": "2021-10-31T01:09:11.106648Z", "custom_metadata": null, "deletion_time": "", "destroyed": false, "version": 2}
Note If you are running Vault 1.9.0 or later, the PATCHverb also supports check-and-set.
To enable the Check-And-Set operation, select the Require Check andSet check-box when you create secrets.
Note UI will always write a new version with cas
even if thecheck-and-set operation is not required.
Q: How do I enter my secrets without exposing the secret in my shell's history?
As a precaution, you may wish to avoid passing your secret as a part of the CLIcommand so that the secret won't appear in the history file. Here are a fewtechniques you can use.
Option 1: Use a dash "-"
An easy technique is to use a dash "-" and then press Enter. This allows you toenter the secret on a new line. After entering the secret, press Ctrl+d
toend the pipe which will write the secret to the Vault.
$ vault kv put kv-v1/eng/apikey/Google key=-AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI<Ctrl+d>
Option 2: Read the secret from a file
The key-value pairs for a secret may be defined in a file.
Create a file named apikey.json
that defines the key
field.
$ tee apikey.json <<EOF{ "key": "AAaaBBccDDeeOTXzSMT1234BB_Z8JzG7JkSVxI"}EOF
Create a secret at path kv-v1/eng/apikey/Google
with keys and values definedin apikey.json
.
$ vault kv put kv-v1/eng/apikey/Google @apikey.json
Option 3: Disable all vault command history
The first two options ensure that the contents of the secret do not appear inthe shell history. The secret path would still be accessible through the shellhistory.
You can configure your shell to avoid logging any vault
commands to yourhistory.
In bash
, set the history to ignore all commands that start with vault
.
$ export HISTIGNORE="&:vault*"
Note This prevents vault
commands from appearing when using the Uparrow or when searching command history with <Ctrl-r>
.
Q: How do I save multiple values at once?
The tutorial examples demonstrating a secret with a single key-value pair.Multiple key-value pairs may be provided in a single command.
Create a secret at path kv-v1/dev/config/mongodb
that sets the url
,db_name
, username
, and password
.
$ vault kv put kv-v1/dev/config/mongodb \ url=foo.example.com:35533 \ db_name=users \ username=admin password=passw0rd
Multiple key-value pairs may be defined in a file provided to the command.
Create a file named mongodb.json
that defines the url
, db_name
, username
,and password
fields.
$ tee mongodb.json <<EOF{ "url": "foo.example.com:35533", "db_name": "users", "username": "admin", "password": "pa$$w0rd"}EOF
Create a secret at path kv-v1/dev/config/mongodb
with keys and values definedin mongodb.json
.
Q: How do I store secrets generated by Vault in KV secrets engine?
Assuming that you have AppRole auth method enabled (refer to theAppRole Pull Authentication tutorial).
The following command retrieves the Secret ID of the jenkins
role, encode itwith base64, and stores the value in the kv-v1/secret-id
path.
$ vault kv put kv-v1/secret-id \ jenkins-secret-id="$(vault write -f -field=secret_id auth/approle/role/jenkins/secret-id | base64)"
Output:
Success! Data written to: kv-v1/secret-id
Next steps
This tutorial demonstrated the versioned KV secrets engine (kv-v2
) feature. Tointegrate the KV secrets engine into your existing application, you mustimplement the Vault API to accomplish that. Alternatively, you can leverageVault Agent which significantly reduces the amount of code change introduced toyour application.
The Vault Agent Templates tutorial providesan end-to-end example.
Help and reference
- KV Secrets Engine - Version 2
- KV Secrets Engine - Version 2 (API)