Migrating from Cloud-Config to Container Linux Config

Historically, the recommended way to provision a Container Linux machine was with a cloud-config. This was a YAML file specifying things like systemd units to run, users that should exist, and files that should be written. This file would be given to a Container Linux machine, and saved on disk. Then a utility called coreos-cloudinit running in a systemd unit would read this file, look at the system state, and make necessary changes on every boot.

Going forward, a new method of provisioning with Container Linux Configs is now recommended.

This document details how to convert an existing cloud-config into a Container Linux Config. Once a Container Linux Config has been written, it is given to the Config Transpiler to be converted into an Ignition Config. This Ignition Config can then be provided to a booting machine. For more information on this process, take a look at the provisioning guide.

The etcd and flannel examples shown in this document will use dynamic data in the Container Linux Config (anything looking like this: {PRIVATE_IPV4}). Not all types of dynamic data are supported on all cloud providers, and if the machine is not on a cloud provider this feature cannot be used. Please see here for more information.

To see all supported options available in a Container Linux Config, please look at the Container Linux Config schema.

etcd2

In a cloud-config, etcd version 2 can be enabled and configured by using the coreos.etcd2.* section. As an example of this:

 #cloud-config

coreos:
  etcd2:
    discovery:                   "https://discovery.etcd.io/<token>"
    advertise-client-urls:       "http://$public_ipv4:2379"
    initial-advertise-peer-urls: "http://$private_ipv4:2380"
    listen-client-urls:          "http://0.0.0.0:2379,http://0.0.0.0:4001"
    listen-peer-urls:            "http://$private_ipv4:2380,http://$private_ipv4:7001"
CoreOS cloud-configs can validated using the online validator.

etcd can be configured in a more general way with a Container Linux Config. This CL Config will use the etcd-member.service systemd unit rather than the etcd2 service understood by cloud-config and coreos-cloudinit. The etcd-member service will download a version of etcd of the user's choosing and run it. This means that in a Container Linux Config both etcd v2 and v3 can be configured.

This is done under the etcd section:

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

etcd:
    version: 3.1.6
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {
    "units": [
      {
        "name": "etcd-member.service",
        "enable": true,
        "dropins": [
          {
            "name": "20-clct-etcd-member.conf",
            "contents": "[Service]\nEnvironment=\"ETCD_IMAGE_TAG=v3.1.6\"\nExecStart=\nExecStart=/usr/lib/coreos/etcd-wrapper $ETCD_OPTS"
          }
        ]
      }
    ]
  },
  "networkd": {},
  "passwd": {}
}

Omitting the version specification declares that the unit file should use the version of etcd matching the running version of Container Linux.

Configuration options in this section can be provided the same way as they were in a cloud-config, with the exception of dashes (-) being replaced with underscores (_) in key names.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

etcd:
  name:                        "{HOSTNAME}"
  advertise_client_urls:       "{PRIVATE_IPV4}:2379"
  initial_advertise_peer_urls: "{PRIVATE_IPV4}:2380"
  listen_client_urls:          "http://0.0.0.0:2379"
  listen_peer_urls:            "http://{PRIVATE_IPV4}:2380"
  initial_cluster:             "%m=http://{PRIVATE_IPV4}:2380"
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {
    "units": [
      {
        "name": "etcd-member.service",
        "enable": true,
        "dropins": [
          {
            "name": "20-clct-etcd-member.conf",
            "contents": "[Unit]\nRequires=coreos-metadata.service\nAfter=coreos-metadata.service\n\n[Service]\nEnvironmentFile=/run/metadata/coreos\nExecStart=\nExecStart=/usr/lib/coreos/etcd-wrapper $ETCD_OPTS \\\n  --name=\"${COREOS_GCE_HOSTNAME}\" \\\n  --listen-peer-urls=\"http://${COREOS_GCE_IP_EXTERNAL_0}:2380\" \\\n  --listen-client-urls=\"http://0.0.0.0:2379\" \\\n  --initial-advertise-peer-urls=\"${COREOS_GCE_IP_EXTERNAL_0}:2380\" \\\n  --initial-cluster=\"%m=http://${COREOS_GCE_IP_EXTERNAL_0}:2380\" \\\n  --advertise-client-urls=\"${COREOS_GCE_IP_EXTERNAL_0}:2379\""
          }
        ]
      }
    ]
  },
  "networkd": {},
  "passwd": {}
}

flannel

Flannel is easily configurable in a cloud-config the same way etcd is, by using the coreos.flannel.* section.

 #cloud-config

coreos:
  flannel:
      etcd_prefix: "/coreos.com/network2"
CoreOS cloud-configs can validated using the online validator.

The flannel section in a Container Linux Config is used the same way, and a version can optionally be specified for flannel as well.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

flannel:
  version:     0.7.0
  etcd_prefix: "/coreos.com/network2"
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {
    "units": [
      {
        "name": "flanneld.service",
        "enable": true,
        "dropins": [
          {
            "name": "20-clct-flannel.conf",
            "contents": "[Service]\nEnvironment=\"FLANNEL_IMAGE_TAG=v0.7.0\"\nExecStart=\nExecStart=/usr/lib/coreos/flannel-wrapper $FLANNEL_OPTS \\\n  --etcd-prefix=\"/coreos.com/network2\""
          }
        ]
      }
    ]
  },
  "networkd": {},
  "passwd": {}
}

locksmith

The coreos.locksmith.* section in a cloud-config can be used to configure the locksmith daemon via environment variables.

 #cloud-config

coreos:
  locksmith:
      endpoint: "http://example.com:2379"
CoreOS cloud-configs can validated using the online validator.

Locksmith can be configured in the same way under the locksmith section of a Container Linux Config, but some of the accepted options are slightly different. Also the reboot strategy is set in the locksmith section, instead of the update section. Check out the Container Linux Config schema to see what options are available.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

locksmith:
  reboot_strategy: "reboot"
  etcd_endpoints:  "http://example.com:2379"
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {
    "files": [
      {
        "filesystem": "root",
        "path": "/etc/coreos/update.conf",
        "contents": {
          "source": "data:,%0AREBOOT_STRATEGY%3D%22reboot%22%0ALOCKSMITHD_ENDPOINTS%3D%22http%3A%2F%2Fexample.com%3A2379%22",
          "verification": {}
        },
        "mode": 420,
        "user": {},
        "group": {}
      }
    ]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}

update

The coreos.update.* section can be used to configure the reboot strategy, update group, and update server in a cloud-config.

 #cloud-config
coreos:
  update:
    reboot-strategy: "etcd-lock"
    group:           "stable"
    server:          "https://public.update.core-os.net/v1/update/"
CoreOS cloud-configs can validated using the online validator.

In the update section in a Container Linux Config the group and server can be configured, but the reboot-strategy option has been moved under the locksmith section.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

update:
  group:  "stable"
  server: "https://public.update.core-os.net/v1/update/"
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {
    "files": [
      {
        "filesystem": "root",
        "path": "/etc/coreos/update.conf",
        "contents": {
          "source": "data:,GROUP%3Dstable%0ASERVER%3Dhttps%3A%2F%2Fpublic.update.core-os.net%2Fv1%2Fupdate%2F",
          "verification": {}
        },
        "mode": 420,
        "user": {},
        "group": {}
      }
    ]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}

units

The coreos.units.* section in a cloud-config can define arbitrary systemd units that should be started after booting.

 #cloud-config

coreos:
  units:
    - name: "docker-redis.service"
      command: "start"
      content: |
        [Unit]
        Description=Redis container
        Author=Me
        After=docker.service

        [Service]
        Restart=always
        ExecStart=/usr/bin/docker start -a redis_server
        ExecStop=/usr/bin/docker stop -t 2 redis_server
CoreOS cloud-configs can validated using the online validator.

This section could also be used to define systemd drop-in files for existing units.

 #cloud-config

coreos:
  units:
    - name: "docker.service"
      drop-ins:
        - name: "50-insecure-registry.conf"
          content: |
            [Service]
            Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
CoreOS cloud-configs can validated using the online validator.

And existing units could also be started without any further configuration.

 #cloud-config

coreos:
  units:
    - name: "etcd2.service"
      command: "start"
CoreOS cloud-configs can validated using the online validator.

One big difference in Container Linux Config compared to cloud-configs is that the configuration is applied via Ignition before the machine has fully booted, as opposed to coreos-cloudinit that runs after the machine has fully booted. As a result units cannot be directly started in a Container Linux Config, the unit is instead enabled so that systemd will begin the unit once systemd starts.

Note: in this example an [Install] section has been added so that the unit can be enabled.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

systemd:
  units:
    - name: "docker-redis.service"
      enable: true
      contents: |
        [Unit]
        Description=Redis container
        Author=Me
        After=docker.service

        [Service]
        Restart=always
        ExecStart=/usr/bin/docker start -a redis_server
        ExecStop=/usr/bin/docker stop -t 2 redis_server

        [Install]
        WantedBy=multi-user.target
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {
    "units": [
      {
        "name": "docker-redis.service",
        "enable": true,
        "contents": "[Unit]\nDescription=Redis container\nAuthor=Me\nAfter=docker.service\n\n[Service]\nRestart=always\nExecStart=/usr/bin/docker start -a redis_server\nExecStop=/usr/bin/docker stop -t 2 redis_server\n\n[Install]\nWantedBy=multi-user.target"
      }
    ]
  },
  "networkd": {},
  "passwd": {}
}

Drop-in files can be provided for units in a Container Linux Config just like in a cloud-config.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

systemd:
  units:
    - name: "docker.service"
      dropins:
        - name: "50-insecure-registry.conf"
          contents: |
            [Service]
            Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {
    "units": [
      {
        "name": "docker.service",
        "dropins": [
          {
            "name": "50-insecure-registry.conf",
            "contents": "[Service]\nEnvironment=DOCKER_OPTS='--insecure-registry=\"10.0.1.0/24\"'"
          }
        ]
      }
    ]
  },
  "networkd": {},
  "passwd": {}
}

Existing units can also be enabled without configuration.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

systemd:
  units:
    - name: "etcd-member.service"
      enable: true
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {
    "units": [
      {
        "name": "etcd-member.service",
        "enable": true
      }
    ]
  },
  "networkd": {},
  "passwd": {}
}

ssh_authorized_keys

In a cloud-config the ssh_authorized_keys section can be used to add ssh public keys to the core user.

 #cloud-config

ssh_authorized_keys:
  - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
CoreOS cloud-configs can validated using the online validator.

In a Container Linux Config there is no analogous section to ssh_authorized_keys, but ssh keys for the core user can be set just as easily using the passwd.users.* section:

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {},
  "networkd": {},
  "passwd": {
    "users": [
      {
        "name": "core",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
        ]
      }
    ]
  }
}

hostname

In a cloud-config the hostname section can be used to set a machine's hostname.

 #cloud-config

hostname: "coreos1"
CoreOS cloud-configs can validated using the online validator.

The Container Linux Config is intentionally more generalized than a cloud-config, and there is no equivalent hostname section understood in a CL Config. Instead, set the hostname by writing it to /etc/hostname in a CL Config storage.files.* section.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

storage:
  files:
    - filesystem: "root"
      path:       "/etc/hostname"
      mode:       0644
      contents:
        inline: coreos1
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {
    "files": [
      {
        "filesystem": "root",
        "path": "/etc/hostname",
        "contents": {
          "source": "data:,coreos1",
          "verification": {}
        },
        "mode": 420,
        "user": {},
        "group": {}
      }
    ]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}

users

The users section in a cloud-config can be used to add users and specify many properties about them, from groups the user should be in to what the user's shell should be.

 #cloud-config

users:
  - name: "elroy"
    passwd: "$6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm..."
    groups:
      - "sudo"
      - "docker"
    ssh-authorized-keys:
      - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
CoreOS cloud-configs can validated using the online validator.

This same information can be added to the Container Linux Config in the passwd.users.* section.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

passwd:
  users:
    - name:          "elroy"
      password_hash: "$6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm..."
      ssh_authorized_keys:
        - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
      create:
        groups:
          - "sudo"
          - "docker"
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {},
  "systemd": {},
  "networkd": {},
  "passwd": {
    "users": [
      {
        "name": "elroy",
        "passwordHash": "$6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h..."
        ],
        "create": {
          "groups": [
            "sudo",
            "docker"
          ]
        }
      }
    ]
  }
}

Note: the create: section must exist or the user will not be created.

write_files

The write_files section in a cloud-config can be used to specify files and their contents that should be written to disk on the machine.

 #cloud-config
write_files:
  - path:        "/etc/resolv.conf"
    permissions: "0644"
    owner:       "root"
    content: |
      nameserver 8.8.8.8
CoreOS cloud-configs can validated using the online validator.

This can be done in a Container Linux Config with the storage.files.* section.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

storage:
  files:
    - filesystem: "root"
      path:       "/etc/resolv.conf"
      mode:       0644
      contents:
        inline: |
          nameserver 8.8.8.8
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {
    "files": [
      {
        "filesystem": "root",
        "path": "/etc/resolv.conf",
        "contents": {
          "source": "data:,nameserver%208.8.8.8",
          "verification": {}
        },
        "mode": 420,
        "user": {},
        "group": {}
      }
    ]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}

File specifications in this section of a CL Config must define the target filesystem and the file's path relative to the root of that filesystem. This allows files to be written to filesystems other than the root filesystem.

Under the contents section, the file contents are under a sub-section called inline. This is because a file's contents can be remote by replacing the inline section with a remote section. To see what options are available under the remote section, look at the Container Linux Config schema.

manage_etc_hosts

The manage_etcd_hosts section in a cloud-config can be used to configure the contents of the /etc/hosts file. Currently only one value is supported, "localhost", which will cause your system's hostname to resolve to 127.0.0.1.

 #cloud-config

manage_etc_hosts: "localhost"
CoreOS cloud-configs can validated using the online validator.

There is no analogous section in a Container Linux Config, however the /etc/hosts file can be written in the storage.files.* section.

This is the human-readable config file. This should not be immediately passed to Container Linux. Learn more.
# This config is meant to be consumed by the config transpiler, which will
# generate the corresponding Ignition config. Do not pass this config directly
# to instances of Container Linux.

storage:
  files:
    - filesystem: "root"
      path:       "/etc/hosts"
      mode:       0644
      contents:
        inline: |
          127.0.0.1	localhost
          ::1		localhost
          127.0.0.1 example.com
This is the raw machine configuration, which is not intended for editing. Learn more. Validate the config here.
{
  "ignition": {
    "version": "2.0.0",
    "config": {}
  },
  "storage": {
    "files": [
      {
        "filesystem": "root",
        "path": "/etc/hosts",
        "contents": {
          "source": "data:,127.0.0.1%09localhost%0A%3A%3A1%09%09localhost%0A127.0.0.1%20example.com",
          "verification": {}
        },
        "mode": 420,
        "user": {},
        "group": {}
      }
    ]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}