Rebooting a 1000 times with kexec and systemd
I've recently been faced with the tedious task of collecting some data, namely the TSC frequency as calibrated by the Linux Kernel, across a number of boots. After 25 manual reboots, I got really bored and worse, I had to recognize that I would need approximately 1000 reboots in order to gain any statistical power.
So I asked myself the question whether this was scriptable. Thanks to
kexec
, it is.
Introduction to kexec
kexec
is a syscall which allows to boot into a given kernel from a
running one. With the userspace /usr/sbin/kexec
, this is done in
to steps:
- Load
- Configure the kernel image, initrd and command line to be
used in the next "execute" step:
# /usr/sbin/kexec -l /boot/vmlinuz-<whatever> \ --initrd=/boot/initramfs-<whatever>.img \ --command-line='some kernel parameters'
- Execute
- Actually boot into the just loaded kernel by running
# /usr/sbin/kexec -e
Integration with systemd
Scripted rebooting
Now, the obvious place to trigger this kexec
stuff from is some
systemd unit. Ideally, we'd like to give the first booted kernel a
parameter like spinningreboot=<n>
and give the reboot loop a go.
To achieve that, do the following:
- Get the
spinning-reboot.sh
, put it into/usr/local/bin
and make it executable.spinning-reboot.sh
will be the workhorse called from the systemd service created below. It extracts the currentspinningreboot
parameter from the current kernel's command line, if any. It checks whether this value is above zero and if so, assembles a new command line mostly equal to the current one but with thespinningreboot
parameter decremented by one. Afterwards it does akexec -l
with the currently running kernel and initrd and boots into it withkexec -e
. For this to work,/boot
is required to be available, hence theRequiresMountFor
dependency from the systemd unit created in the next step. - Create a
/etc/systemd/system/spinning-reboot.service
with the following contents[Unit] Description=Spinning reboot RequiresMountsFor=/boot ConditionKernelCommandLine=|spinningreboot [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/local/bin/spinning-reboot.sh [Install] WantedBy=basic.target
- Enable the new service by means of
# systemctl enable spinning-reboot.service'
Since
spinning-reboot.sh
does nothing if there isn't anyspinningreboot
kernel parameter present, it should be safe to keep this enabled forever.
Doing something useful in between reboots
Now that we are able to reboot like hell without any manual
intervention, we'd like to something useful before rebooting like
collecting data or whatever. The solution is to add another systemd
service carrying out the desired action. In order to get it run
before the reboot is triggered, its unit file should have a
Before=spinning-reboot.service
ordering constraint.
Furthermore, it should have dependencies on the systemd services it
needs. For example, if you need your local /etc/fstab
filesystems
mounted, a dependency on the local-fs.target
will suffice. Since
dependencies and ordering constraints are orthogonal with systemd,
you'll also need an After
clause on the dependency
required. Alternatively, you could use a RequiresMountFor
clause.
For example, consider the following data collection script I put in
/usr/local/bin/record-tsc-freq.sh
:
#!/bin/sh processorfreq=$(dmesg | sed '/tsc: Detected/ {s/.* \([0-9.]\+\) MHz processor$/\1/; p}; d') refinedfreq=$(dmesg | sed '/tsc: Refined TSC clocksource calibration/ {s/.*: \([0-9.]\+\) MHz$/\1/; p}; d') echo "$processorfreq $refinedfreq" >> /mnt/scratch/nic/tscfreqs.csv
It extracts the TSC frequencies as calibrated by the kernel from
the dmesg
output and writes them into
/mnt/scratch/nic/tscfreqs.csv
.
Obviously, /mnt/scratch/nic/
must be mounted when this script is run.
So, there are two possibilities for a systemd unit file
- Dependency on systemd's
local-fs.target
[Unit] Description=TSC frequency recording Requires=local-fs.target After=local-fs.target Before=spinning-reboot.service ConditionKernelCommandLine=|spinningreboot [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/local/bin/record-tsc-freq.sh [Install] WantedBy=basic.target
- Using systemd's
RequiresMountFor
dependency clause[Unit] Description=TSC frequency recording RequiresMountFor=/mnt/scratch/nic Before=spinning-reboot.service ConditionKernelCommandLine=|spinningreboot [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/local/bin/record-tsc-freq.sh [Install] WantedBy=basic.target
Either way, put the content into a
/etc/systemd/system/record-tsc-freq.service
and enable it via
systemctl enable record-tsc-freq.service
.
Hint: reducing cycle times
Systemd aggressively starts the services needed to reach its final
target like graphical.target
. For me, it proved helpful to tell
systemd not to hunt for the full blown graphical.target
, but for
basic.target
only. This can be communicated via the
systemd.unit=<target>
kernel parameter to systemd.