Setting up SSH access and starting containers at boot

Complete all the steps described in the initial setup page before starting this tutorial. Make sure you have create file ssh-key.pub following the instructions provided in the prerequisites for the tutorial. We will use this key in the Butane configuration file that we are about to write.

In this tutorial, we will set up SSH access and start a container at boot. Fedora CoreOS is focused on running applications/services in containers thus we recommend trying to run containers and avoid modifying the host directly. Running containers and keeping a pristine host layer makes automatic updates more reliable and allows for separation of concerns with the Fedora CoreOS team responsible for the OS and end-user operators/sysadmins responsible for the applications.

As usual, we will set up console autologin, a hostname, systemd pager configuration, but we will also:

Add an SSH Key for the core user from the local ssh-key.pub file.
Add a systemd service (failure.service) that fails on boot.
Add a running container via a Podman quadlet systemd.unit container file.
This etcd-member.container will then be associated with a etcd-member.service on the running system.
etcd-member.service will launch and manage the lifecycle of the container using podman.

Writing the Butane config and converting to Ignition

Similarly to what we did in the second provisioning scenario, we will write the following Butane config in a file called containers.bu:

Example with automatic serial console login, SSH key, and multiple systemd units

variant: fcos
version: 1.5.0
passwd:
  users:
    - name: core
      ssh_authorized_keys_local:
        - ssh-key.pub
systemd:
  units:
    - name: [email protected]
      dropins:
      - name: autologin-core.conf
        contents: |
          [Service]
          # Override Execstart in main unit
          ExecStart=
          # Add new Execstart with `-` prefix to ignore failure
          ExecStart=-/usr/sbin/agetty --autologin core --noclear %I $TERM
          TTYVTDisallocate=no
    - name: failure.service
      enabled: true
      contents: |
        [Service]
        Type=oneshot
        ExecStart=/usr/bin/false
        RemainAfterExit=yes

        [Install]
        WantedBy=multi-user.target
storage:
  files:
    - path: /etc/hostname
      mode: 0644
      contents:
        inline: |
          tutorial
    - path: /etc/profile.d/systemd-pager.sh
      mode: 0644
      contents:
        inline: |
          # Tell systemd to not use a pager when printing information
          export SYSTEMD_PAGER=cat
    - path: /etc/containers/systemd/etcd-member.container
      mode: 0644
      contents:
        inline: |
          [Unit]
          Description=Run a single node etcd
          After=network-online.target
          Wants=network-online.target

          [Container]
          ContainerName=etcd
          Image=quay.io/coreos/etcd:latest
          Network=host
          Volume=etcd-data:/etcd-data
          Exec=/usr/local/bin/etcd                                \
              --name node1 --data-dir /etcd-data                  \
              --initial-advertise-peer-urls http://127.0.0.1:2380 \
              --listen-peer-urls http://127.0.0.1:2380            \
              --advertise-client-urls http://127.0.0.1:2379       \
              --listen-client-urls http://127.0.0.1:2379          \
              --initial-cluster node1=http://127.0.0.1:2380

          [Install]
          WantedBy=multi-user.target

Run Butane to convert that to an Ignition config:

butane --pretty --strict --files-dir=./ containers.bu --output containers.ign

Now let’s provision it:

# Setup the correct SELinux label to allow access to the config
chcon --verbose --type svirt_home_t containers.ign

# Start a Fedora CoreOS virtual machine
virt-install --name=fcos --vcpus=2 --ram=2048 --os-variant=fedora-coreos-stable \
    --import --network=bridge=virbr0 --graphics=none \
    --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${PWD}/containers.ign" \
    --disk=size=20,backing_store=${PWD}/fedora-coreos.qcow2

On the console you will see:

Fedora CoreOS 38.20230709.3.0
Kernel 6.3.11-200.fc38.x86_64 on an x86_64 (ttyS0)

SSH host key: SHA256:T5V4oXMZ0UJ7WRGzNiUOkggO7p5yojTVBUxa6N3vIoQ (ECDSA)
SSH host key: SHA256:oBAvj2kaKKKK++gnchTbxpp/iphvX6EHr0EynwXZ19c (ED25519)
SSH host key: SHA256:Yg2fdA7GC1eoHtIjawDA+WffTKTuNy5ZhQHUJx5GRHk (RSA)
enp1s0: 192.168.124.119 fe80::9b5c:330d:2020:1c9e
Ignition: ran on 2023/08/03 18:17:45 UTC (this boot)
Ignition: user-provided config was applied
Ignition: wrote ssh authorized keys file for user: core
tutorial login: core (automatic login)

Fedora CoreOS 38.20230709.3.0
[systemd]
Failed Units: 1
  failure.service
[core@tutorial ~]$

If you would like to connect via SSH, disconnect from the serial console by pressing CTRL + ] and then use the reported IP address for the NIC from the serial console to log in using the core user via SSH:

$ ssh [email protected]
The authenticity of host '192.168.124.119 (192.168.124.119)' can't be established.
ED25519 key fingerprint is SHA256:oBAvj2kaKKKK++gnchTbxpp/iphvX6EHr0EynwXZ19c.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.124.119' (ED25519) to the list of known hosts.
Fedora CoreOS 38.20230709.3.0
Tracker: https://github.com/coreos/fedora-coreos-tracker
Discuss: https://discussion.fedoraproject.org/tag/coreos

Last login: Thu Aug  3 18:18:06 2023
[systemd]
Failed Units: 1
  failure.service

The Failed Units message is coming from the console login helper messages helpers. This particular helper shows us when systemd has services that are in a failed state. In this case we made failure.service with ExecStart=/usr/bin/false, so we intentionally created a service that will always fail in order to illustrate the helper messages.

Now that we’re up and don’t have any real failures we can check out the status of etcd-member.service, which was generated from our etcd-member.container file.

[core@tutorial ~]$ systemctl status --full etcd-member.service
● etcd-member.service - Run a single node etcd
     Loaded: loaded (/etc/containers/systemd/etcd-member.container; generated)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Thu 2023-08-03 18:17:57 UTC; 2min 24s ago
   Main PID: 1553 (conmon)
      Tasks: 10 (limit: 2238)
     Memory: 86.5M
        CPU: 3.129s
     CGroup: /system.slice/etcd-member.service
             ├─libpod-payload-31af97b0ef902b3b3b3d717bd98947b209701b9585db2129ca53f4b33962415e
             │ └─1555 /usr/local/bin/etcd ...
             └─runtime
               └─1553 /usr/bin/conmon ...

Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.745207 I | raft: b71f75320dc06a6c became candidate at term 2
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.745372 I | raft: b71f75320dc06a6c received MsgVoteResp from b71f75320dc06a6c at term 2
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.745499 I | raft: b71f75320dc06a6c became leader at term 2
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.745628 I | raft: raft.node: b71f75320dc06a6c elected leader b71f75320dc06a6c at term 2
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.746402 I | etcdserver: setting up the initial cluster version to 3.3
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.747906 N | etcdserver/membership: set the initial cluster version to 3.3
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.748211 I | etcdserver/api: enabled capabilities for version 3.3
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.748384 I | etcdserver: published {Name:node1 ClientURLs:[http://127.0.0.1:2379]} to cluster 1c45a069f3a1d796
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.748510 I | embed: ready to serve client requests
Aug 03 18:17:58 tutorial etcd[1553]: 2023-08-03 18:17:58.750778 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!

We can also inspect the state of the container that was run by the systemd service:

[core@tutorial ~]$ sudo podman ps -a
CONTAINER ID  IMAGE                       COMMAND               CREATED        STATUS        PORTS       NAMES
31af97b0ef90  quay.io/coreos/etcd:latest  /usr/local/bin/et...  4 minutes ago  Up 4 minutes              etcd

And we can set a key/value pair in etcd. For now let’s set the key fedora to the value fun:

[core@tutorial ~]$ curl -L -X PUT http://127.0.0.1:2379/v2/keys/fedora -d value="fun"
{"action":"set","node":{"key":"/fedora","value":"fun","modifiedIndex":4,"createdIndex":4}}
[core@tutorial ~]$ curl -L http://127.0.0.1:2379/v2/keys/ 2>/dev/null | jq .
{
  "action": "get",
  "node": {
    "dir": true,
    "nodes": [
      {
        "key": "/fedora",
        "value": "fun",
        "modifiedIndex": 4,
        "createdIndex": 4
      }
    ]
  }
}

Looks like everything is working!

Cleanup

Now let’s take down the instance for the next test. Disconnect from the serial console by pressing CTRL + ] or from SSH and then destroy the machine:

virsh destroy fcos
virsh undefine --remove-all-storage fcos

You may now proceed with the next tutorial.

Want to help? Learn how to contribute to Fedora Docs ›