How to properly schedule SCRUB for more than 2 ZFS pools
How to properly schedule SCRUB for more than 2 ZFS pools

How to properly schedule SCRUB for more than 2 ZFS pools

A scrub is a maintenance process that systematically reads all data and every block of data in a ZFS storage pool and verifies its integrity using checksums. Every block of data in ZFS file system has a checksum. During a scrub, it reads all the data, recalculates the checksum and compares it with the stored checksum. It is a crucial feature of ZFS’s self healing architecture designed to detect and automatically repair silent data corruption (bit rot) before it accumulates and becomes unrecoverable. Scrub is also an early warning system for your hardware. Frequent checksum errors during a scrub are a strong indicator that a drive is failing, often allowing you to replace it before it completely dies. Without scrubs, you might think your data is fine but corruption could already exist unnoticed. Scrub will give you confidence that your data is still good.

You can manually run scrub by issuing command in terminal like this. The more data you have the longer it will take to finish the process depending how fast your drives are and how fast your system is calculating all the checksum.

sudo zpool scrub <pool-name>

If you happen to run TrueNAS OS, scrub is automatically scheduled to run every week happening on Sunday at 12:00 AM (midnight) for every new pool created in the operating system.

How often scrub is run really depends how high-churn (heavy writing/reading) your system is or older your drives are. At the very least, once a month schedule is often recommended and sufficient for most environments. Remember the impact of your system during a scrub process is high (drives often get hot) even though scrubs are low-priority background tasks. But, they do cause high I/O load and read data, so schedule them during low-usage hours.

In this post, I will show you how I schedule a scrub for all 3 of my ZFS pools that I managed in my Ubuntu 24.04 LTS using system timer. Assuming, I have 3 ZFS pools with following names tank, cloud and evo-pool I want to run scrub monthly for each pool but each scrub will be 1 week apart happening only on Tuesday at 4:00AM. By doing this, I don’t stress my entire system running scrub all on my ZFS pools at once.

1. Create Template Service File

First, I copy a default template scrub service from /lib/systemd/system/ into /etc/systemd/system/ This template approach is generally considered “best practice” because it allows you to schedule different pools at different times (e.g., scrubbing your fast SSD pool weekly and your massive HDD archive monthly). The @ in the filename is what allows it to accept the pool name as an argument.

sudo cp /lib/systemd/system/[email protected] /etc/systemd/system/[email protected]

The content on the newly created service file [email protected] should be something like this:

[Unit]
Description=zpool scrub on %i
Documentation=man:zpool-scrub(8)
Requires=zfs.target
After=zfs.target
ConditionACPower=true
ConditionPathIsDirectory=/sys/module/zfs

[Service]
EnvironmentFile=-/etc/default/zfs
ExecStart=sh -c '\
if zpool status %i | grep -q "scrub in progress"; then \
exec zpool wait -t scrub %i; \
else exec zpool scrub -w %i; fi'
ExecStop=-sh -c 'zpool scrub -p %i 2>/dev/null || true'
2. Create the Timer File

To handle all 3 pools on a staggered schedule (one per week), the timer strategy changes. Since I want a specific sequence (1st, 2nd and 3rd Tuesday of the month), creating 3 individual timer files is the cleanest way to manage them. This allows me to hard-code the specific Tuesday for each pool

sudo nano /etc/systemd/system/zfs-scrub-tank.timer
sudo nano /etc/systemd/system/zfs-scrub-cloud.timer
sudo nano /etc/systemd/system/zfs-scrub-evo.timer
PoolFile NameOnCalendar Value
tankzfs-scrub-tank.timerTue *-*-1..7 (1st Tue)
cloudzfs-scrub-cloud.timerTue *-*-8..14 (2nd Tue)
evo-poolzfs-scrub-evo.timerTue *-*-15..21 (3rd Tue)

Example timer configuration: For each file, you will point the Unit= line to your template service, specifying the pool name after the @ and set the timer OnCalendar= line to specific timer you want. RandomizedDelaySec=1800 means that scrub process will randomly start at time between 4:00AM and 4:30AM. (1800s = 30min). In simple terms, Persistent=true is an “insurance policy” for your schedule. It ensures that if your computer is powered off when a timer is supposed to fire, the task will run immediately when you turn the computer back on. Without it, you would have to wait an entire month for the next schedule. Content of example timer file for evo-pool as following:

[Unit]
Description=Scrub evo-pool on the 3rd Tuesday of every month

[Timer]
# 3rd Tuesday is always between the 15th and 21st and start at 4:00AM
OnCalendar=Tue *-*-15..21 04:00:00
# Random delay up to 30 minutes
RandomizedDelaySec=1800
# Ensure it runs even if the machine was off during set timer
Persistent=true
# This tells the timer which pool to start.
[email protected]

[Install]
WantedBy=timers.target

For cloud pool, you can change the OnCalendar= and Unit= lines to corresponding pool service like this

OnCalendar=Tue *-*-8..14 04:00:00
[email protected]

Why this is better for 3 pools: By staggering them by week, you ensure your CPU and disk I/O are never slammed by two scrubs at once. No overlap and easy maintenance. If you decide evo-pool (SSDs/NVMe) should run every week on Wed at 2:00AM instead of once a month, you only change its specific timer file without affecting the others like this OnCalendar=Wed *-*-* 02:00:00. If you want to run scrub on both 1st and 15th day of every month at 2:00AM regardless of what day of the week it is, change OnCalendar=*-*-01,15 02:00:00 Systemd Calendar Timers on Ubuntu is a powerful feature to setup automation for your system.

3. Activation

After creating all 3 .timer files, run these commands to apply the new setup and clear out any old timers:

sudo systemctl daemon-reload
sudo systemctl enable --now zfs-scrub-tank.timer zfs-scrub-cloud.timer zfs-scrub-evo.timer

Check the schedule to make sure they look correct:

systemctl list-timers zfs-scrub-*

Output of list-timers should look something like this:

4. Final Check

After one of the timers starts firing, run this command zpool status to see the current status of all the pools. You should see “scan: scrub repaired 0B in…” with a timestamp matching the timer you specify for that particular ZFS pool. This is a text-book, safe and efficient scrub scheduling practice for ZFS. I hope this article helps you to setup and schedule scrub for all of your ZFS pools intelligently and efficiently. Additional improvement to this scrub scheduling would be setting up Ubuntu 24.04 LTS automation to run smartctl on every drives inside the pools using the same systemd timers and a bash script. Maybe, next time! 😊

Leave a Reply

Your email address will not be published. Required fields are marked *