This document shows how to safely upgrade the operator to a desired version while preserving the cluster's state and data whenever possible. It is assumed that the preexisting cluster is configured to create and store backups to persistent storage. See the backup config guide for details.
First create a backup of your current cluster before starting the upgrade process. See the backup service guide on how to create a backup.
In the case of an upgrade failure you can restore your cluster to the previous state from the previous backup. See the spec examples on how to do that.
An in-place update can be performed when the upgrade is compatible, i.e we can upgrade the operator without affecting the cluster.
To upgrade an operator deployment the image field
spec.template.spec.containers.image needs to be changed via an in-place update.
Change the image field to
vX.Y.Z is the desired version.
$ kubectl edit deployment/etcd-operator # make the image change in your editor then save and close the file
In the case of an incompatible upgrade, the process requires restoring a new cluster from backup. See the incompatible upgrade guide for more information.
v0.6.1+ the operator will no longer create a storage class specified by
--pv-provisioner by default. This behavior is set by the new flag
--create-storage-class which by default is
Note: If your cluster does not have the following backup policy then you can simply upgrade the operator to the
Backup policy that has
pv.storageClass is unset. For example:
spec: backup: backupIntervalInSecond: 30 maxBackups: 5 pv: storageClass: "" volumeSizeInMB: 512 storageType: PersistentVolume
So if your cluster has the above backup policy then do the following steps before upgrading the operator image to
Confirm the name of the storage class for a given cluster:
kubectl -n <namespace> get pvc -l=etcd_cluster=<cluster-name> -o yaml | grep storage-class
spec.backup.pv.storageClassfield to the name of the existing storage class from the previous step.
The v0.6.0 release removes operator S3 flag.
Note: if your cluster is not using operator S3 flag, then you just need recreate etcd-operator deployment with the
If your cluster is using operator S3 flag for backup and want to use S3 backup in v0.6.0, then you need to migrate your cluster to use cluster level backup.
Steps for migration:
Create a backup of you current data using the backup service guide.
Create a new AWS secret
<aws-secret> for cluster level backup:
$ kubectl -n <namespace> create secret generic <aws-secret> --from-file=$AWS_DIR/credentials --from-file=$AWS_DIR/config
Change the backup field in your existing cluster backup spec to enable cluster level backup:
backup: backupIntervalInSecond: 30 maxBackups: 5 storageType: "S3"
backup: backupIntervalInSecond: 30 maxBackups: 5 storageType: "S3" s3: s3Bucket: <your-s3-bucket> awsSecret: <aws-secret>
Apply the cluster backup spec:
$ kubectl -n <namespace> apply -f <your-cluster-deployment>.yaml
Update deployment spec to use
etcd-operator:v0.6.0 and remove the dependency on operator level S3 backup flags:
spec: containers: - name: etcd-operator image: quay.io/coreos/etcd-operator:v0.5.2 command: - /usr/local/bin/etcd-operator - --backup-aws-secret=aws - --backup-aws-config=aws - --backup-s3-bucket=<your-s3-bucket>
spec: containers: - name: etcd-operator image: quay.io/coreos/etcd-operator:v0.6.0 command: - /usr/local/bin/etcd-operator
Apply the updated deployment spec:
$ kubectl -n <namespace> apply -f <your-etcd-operator-deployment>.yaml
0.4.x versions, please update to
Kubernetes cluster version must be 1.7+.
kubectl -n <namespace> delete cluster <cluster-name>
kubectl delete thirdpartyresource cluster.etcd.coreos.com
ClusterRolewith the new rules for CRD.
apiVersion: "etcd.database.coreos.com/v1beta2" kind: "EtcdCluster" metadata: name: <cluster-name> spec: size: <cluster-size> version: "3.1.8" backup: # The same backup spec used to save the backup of the previous cluster . . . . . . restore: backupClusterName: <previous-cluster-name> storageType: <storage-type-of-backup-spec>
The two points of interest in the above CR spec are:
kindfields have been changed as mentioned in the
spec.restorefield needs to be specified according your backup configuration. See spec examples guide on how to specify the
spec.restorefield for your particular backup configuration.
See the release notes of
v0.4.0 for noticeable changes: https://github.com/coreos/etcd-operator/releases/tag/v0.4.0
To upgrade to
v0.3.x the current operator verison must first be upgraded to
v0.2.6+, since versions <
v0.2.6 are not compatible with
Spec.Backup.MaxBackups < 0, previously it had no effect. Now it will get rejected. Please remove it.
Spec.Backup.MaxBackups == 0, previously it had no effect. Now it will create backup sidecar that allows unlimited backups. If you don't want backup sidecar, just don't set any backup policy.