Links#
https://kubernetes.io/docs/concepts/configuration/secret/
https://docs.docker.com/compose/how-tos/environment-variables/
https://docs.aws.amazon.com/eks/latest/userguide/manage-secrets.html
https://cloud.google.com/secret-manager/docs/secret-manager-managed-csi-component
https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-driver
https://www.alibabacloud.com/help/en/kms/key-management-service/user-guide/integrate-kms-secrets-into-ack
https://external-secrets.io/latest/
https://developer.hashicorp.com/vault/docs/platform/k8s/vso
1. Important Points#
configuration goal:
same image
different runtime config
no secret in git
no secret baked into image
fail fast when config missing
config types:
non-secret config:
PORT
LOG_LEVEL
FEATURE_FLAG
PAYMENT_API_URL
secret config:
DATABASE_PASSWORD
JWT_SECRET
API_TOKEN
PRIVATE_KEY
rule:
non-secret -> ConfigMap / env / config file
secret -> Secret / external secret manager / mounted file
2. Application Config Loader#
import "dotenv/config";
import { readFileSync, existsSync } from "node:fs";
import { z } from "zod";
const configSchema = z.object({
NODE_ENV: z.enum(["local", "dev", "test", "staging", "prod"]).default("local"),
PORT: z.coerce.number().int().positive().default(3000),
LOG_LEVEL: z.enum(["debug", "info", "warn", "error"]).default("info"),
DATABASE_URL: z.string().url(),
JWT_SECRET: z.string().min(32)
});
function readSecret(name: string): string | undefined {
const filePath = process.env[`${name}_FILE`];
if (filePath !== undefined && existsSync(filePath)) {
return readFileSync(filePath, "utf8").trim();
}
return process.env[name];
}
const parsed = configSchema.safeParse({
...process.env,
JWT_SECRET: readSecret("JWT_SECRET")
});
if (!parsed.success) {
console.error(parsed.error.flatten().fieldErrors);
process.exit(1);
}
export const config = parsed.data;
支持两种 secret 输入:
JWT_SECRET:
environment variable
JWT_SECRET_FILE:
mounted file path
好处:
Docker env works
Docker env_file works
Kubernetes Secret env works
CSI mounted secret file works
3. Local#
.env.example#
NODE_ENV=local
PORT=3000
LOG_LEVEL=debug
DATABASE_URL=postgres://order:password@localhost:5432/order
JWT_SECRET=change-me-change-me-change-me-change-me
.env.local#
NODE_ENV=local
PORT=3000
LOG_LEVEL=debug
DATABASE_URL=postgres://order:local-password@localhost:5432/order
JWT_SECRET=local-secret-local-secret-local-secret
git rule:
commit .env.example
do not commit .env.local
do not commit real .env.production
4. Docker#
environment variables#
docker run --rm \
-p 3000:3000 \
-e NODE_ENV=prod \
-e PORT=3000 \
-e LOG_LEVEL=info \
-e DATABASE_URL='postgres://order:password@db:5432/order' \
-e JWT_SECRET='change-me-change-me-change-me-change-me' \
order-api:local
env file#
docker run --rm \
-p 3000:3000 \
--env-file .env.local \
order-api:local
mounted secret file#
docker run --rm \
-p 3000:3000 \
-e JWT_SECRET_FILE=/run/secrets/jwt_secret \
-v "$PWD/secrets/jwt_secret:/run/secrets/jwt_secret:ro" \
--env-file .env.local \
order-api:local
recommendation:
local dev:
env_file is simple
production:
platform should inject env or mounted secret
do not keep .env production file on developer machine
5. Kubernetes#
ConfigMap#
apiVersion: v1
kind: ConfigMap
metadata:
name: order-api-config
data:
NODE_ENV: prod
PORT: "3000"
LOG_LEVEL: info
DATABASE_URL: postgres://order:password@postgres:5432/order
Secret#
apiVersion: v1
kind: Secret
metadata:
name: order-api-secret
type: Opaque
stringData:
JWT_SECRET: change-me-change-me-change-me-change-me
envFrom#
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-api
spec:
replicas: 2
selector:
matchLabels:
app: order-api
template:
metadata:
labels:
app: order-api
spec:
containers:
- name: order-api
image: order-api:1.0.0
ports:
- containerPort: 3000
envFrom:
- configMapRef:
name: order-api-config
- secretRef:
name: order-api-secret
secret as mounted file#
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-api
spec:
template:
spec:
containers:
- name: order-api
image: order-api:1.0.0
env:
- name: JWT_SECRET_FILE
value: /run/secrets/JWT_SECRET
volumeMounts:
- name: app-secrets
mountPath: /run/secrets
readOnly: true
volumes:
- name: app-secrets
secret:
secretName: order-api-secret
Kubernetes notes:
env var secret is easy
mounted file secret works better with CSI/external secret providers
application may need restart to reload env var changes
mounted file can be rotated by provider, but app must reread or restart
6. AWS / EKS#
options:
Kubernetes Secret:
simple
secret stored in cluster etcd
AWS Secrets Manager + ASCP + Secrets Store CSI Driver:
mount Secrets Manager secret as file in pod
good for centralized secret management
External Secrets Operator:
sync AWS Secrets Manager into Kubernetes Secret
app SDK:
app reads Secrets Manager directly
more app code and IAM handling
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: order-api-aws-secrets
spec:
provider: aws
parameters:
objects: |
- objectName: "prod/order-api/jwt"
objectType: "secretsmanager"
jmesPath:
- path: "JWT_SECRET"
objectAlias: "JWT_SECRET"
volumes:
- name: secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: order-api-aws-secrets
EKS recommendation:
use IRSA / Pod Identity for workload identity
grant least privilege to read only required secret ARNs
prefer mounted file for secret value
7. GCP / GKE#
options:
Kubernetes Secret:
simple cluster-native secret
Google Secret Manager add-on / CSI:
mount Secret Manager value as file
uses Workload Identity Federation for GKE
External Secrets Operator:
sync Google Secret Manager into Kubernetes Secret
app SDK:
app reads Secret Manager directly
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: order-api-gcp-secrets
spec:
provider: gcp
parameters:
secrets: |
- resourceName: "projects/my-project/secrets/JWT_SECRET/versions/latest"
path: "JWT_SECRET"
GKE recommendation:
use Workload Identity Federation
avoid static service account key files
mount secret as file or sync through External Secrets Operator
8. Azure / AKS#
options:
Kubernetes Secret:
simple cluster-native secret
Azure Key Vault provider for Secrets Store CSI Driver:
mount Key Vault secret as CSI volume
External Secrets Operator:
sync Azure Key Vault into Kubernetes Secret
app SDK:
app reads Key Vault directly
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: order-api-azure-secrets
spec:
provider: azure
parameters:
usePodIdentity: "false"
clientID: "<managed-identity-client-id>"
keyvaultName: "<key-vault-name>"
tenantId: "<tenant-id>"
objects: |
array:
- |
objectName: JWT_SECRET
objectType: secret
AKS recommendation:
use managed identity / workload identity
use Key Vault provider when org standardizes on Key Vault
enable rotation if the app can handle it
9. Alibaba Cloud / ACK#
options:
Kubernetes Secret:
simple cluster-native secret
ACK secret manager / KMS integration:
integrate KMS secrets into ACK
External Secrets Operator:
check current provider support before choosing
app SDK:
app reads Alibaba Cloud KMS Secrets Manager directly
ACK recommendation:
use RAM role / workload identity style access where available
grant least privilege to required KMS secret
prefer provider-supported mounted secret or synced Kubernetes Secret
10. Vault#
options:
Vault Agent Injector:
inject sidecar / template secrets into file
Vault Secrets Operator:
sync Vault secrets to Kubernetes Secrets
External Secrets Operator:
can read from Vault and create Kubernetes Secret
app SDK:
app reads Vault directly
Vault recommendation:
use Kubernetes auth
short TTL dynamic secrets when possible
mount as file for apps that support reload
document rotation behavior
11. Decision#
| Runtime |
Simple |
Better For Production |
Notes |
| Local |
.env.local |
local secret manager if needed |
do not commit local env |
| Docker |
--env-file |
platform env / mounted secret |
do not bake env into image |
| K8S |
ConfigMap + Secret |
CSI / External Secrets |
env requires restart to update |
| EKS |
Kubernetes Secret |
AWS Secrets Manager + CSI / ESO |
use IRSA / Pod Identity |
| GKE |
Kubernetes Secret |
Secret Manager + CSI / ESO |
use Workload Identity |
| AKS |
Kubernetes Secret |
Key Vault CSI / ESO |
use managed identity |
| ACK |
Kubernetes Secret |
KMS integration / provider |
verify current provider support |
| Vault |
mounted file |
Vault Agent / VSO / ESO |
good for multi-cloud |
12. Production Checklist#
application:
validate config at startup
support env var and *_FILE for secrets
fail fast when required secret missing
do not log config values
container:
no .env in image
no secret in Dockerfile ARG / ENV
use runtime injection
kubernetes:
use ConfigMap for non-secret config
use Secret/external secret for sensitive values
restrict RBAC to secrets
enable encryption at rest for cluster secrets
document rotation behavior
cloud:
use workload identity
least privilege secret read
audit secret access
rotate secrets