Rebooting a 1000 times with kexec and systemd

I've recently been faced with the tedious task of collecting some data, namely the TSC frequency as calibrated by the Linux Kernel, across a number of boots. After 25 manual reboots, I got really bored and worse, I had to recognize that I would need approximately 1000 reboots in order to gain any statistical power.

So I asked myself the question whether this was scriptable. Thanks to kexec, it is.

Introduction to `kexec`

kexec is a syscall which allows to boot into a given kernel from a running one. With the userspace /usr/sbin/kexec, this is done in to steps:

Load

Configure the kernel image, initrd and command line to be used in the next "execute" step:

# /usr/sbin/kexec -l /boot/vmlinuz-<whatever> \
		  --initrd=/boot/initramfs-<whatever>.img \
		  --command-line='some kernel parameters'

Execute

Actually boot into the just loaded kernel by running

# /usr/sbin/kexec -e

Integration with systemd

Scripted rebooting

Now, the obvious place to trigger this kexec stuff from is some systemd unit. Ideally, we'd like to give the first booted kernel a parameter like spinningreboot=<n> and give the reboot loop a go.

To achieve that, do the following:

Get the spinning-reboot.sh, put it into /usr/local/bin and make it executable. spinning-reboot.sh will be the workhorse called from the systemd service created below. It extracts the current spinningreboot parameter from the current kernel's command line, if any. It checks whether this value is above zero and if so, assembles a new command line mostly equal to the current one but with the spinningreboot parameter decremented by one. Afterwards it does a kexec -l with the currently running kernel and initrd and boots into it with kexec -e. For this to work, /boot is required to be available, hence the RequiresMountFor dependency from the systemd unit created in the next step.

Create a /etc/systemd/system/spinning-reboot.service with the following contents

[Unit]
Description=Spinning reboot
RequiresMountsFor=/boot
ConditionKernelCommandLine=|spinningreboot

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/spinning-reboot.sh

[Install]
WantedBy=basic.target

Enable the new service by means of
```
# systemctl enable spinning-reboot.service'
```
Since spinning-reboot.sh does nothing if there isn't any spinningreboot kernel parameter present, it should be safe to keep this enabled forever.

Doing something useful in between reboots

Now that we are able to reboot like hell without any manual intervention, we'd like to something useful before rebooting like collecting data or whatever. The solution is to add another systemd service carrying out the desired action. In order to get it run before the reboot is triggered, its unit file should have a Before=spinning-reboot.service ordering constraint.

Furthermore, it should have dependencies on the systemd services it needs. For example, if you need your local /etc/fstab filesystems mounted, a dependency on the local-fs.target will suffice. Since dependencies and ordering constraints are orthogonal with systemd, you'll also need an After clause on the dependency required. Alternatively, you could use a RequiresMountFor clause.

For example, consider the following data collection script I put in /usr/local/bin/record-tsc-freq.sh:

#!/bin/sh

processorfreq=$(dmesg | sed '/tsc: Detected/ {s/.* \([0-9.]\+\) MHz processor$/\1/; p}; d')
refinedfreq=$(dmesg | sed '/tsc: Refined TSC clocksource calibration/ {s/.*: \([0-9.]\+\) MHz$/\1/; p}; d')
echo "$processorfreq $refinedfreq" >> /mnt/scratch/nic/tscfreqs.csv

It extracts the TSC frequencies as calibrated by the kernel from the dmesg output and writes them into /mnt/scratch/nic/tscfreqs.csv.

Obviously, /mnt/scratch/nic/ must be mounted when this script is run. So, there are two possibilities for a systemd unit file

Dependency on systemd's local-fs.target

[Unit]
Description=TSC frequency recording
Requires=local-fs.target
After=local-fs.target
Before=spinning-reboot.service
ConditionKernelCommandLine=|spinningreboot

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/record-tsc-freq.sh

[Install]
WantedBy=basic.target

Using systemd's RequiresMountFor dependency clause

[Unit]
Description=TSC frequency recording
RequiresMountFor=/mnt/scratch/nic
Before=spinning-reboot.service
ConditionKernelCommandLine=|spinningreboot

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/record-tsc-freq.sh

[Install]
WantedBy=basic.target

Either way, put the content into a /etc/systemd/system/record-tsc-freq.service and enable it via systemctl enable record-tsc-freq.service.

Hint: reducing cycle times