libzfs: honor literal for vdev fragmentation property
"zpool get -p fragmentation <pool> all-vdevs" printed the value with
a trailing '%', unlike the pool fragmentation property and the vdev
capacity property. Print the bare number when literal is requested.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #19222
vdev_prop_get: report real vdev fragmentation
VDEV_PROP_FRAGMENTATION was taken from vd->vdev_stat.vs_fragmentation,
which is never updated: fragmentation is only filled into the copy
returned by vdev_get_stats_ex(). So "zpool get fragmentation <pool>
all-vdevs" always reported 0%.
Report the same values as "zpool list -v": pool fragmentation for the
root vdev, metaslab group fragmentation for top-level concrete vdevs,
and "-" for everything else.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #16459
Closes #19222
dbuf: restore BP_IS_HOLE() recheck after dnode_block_freed()
Commit 79cf54582 ("Fix reads for blocks freed after being cloned")
reworked the level-0 check in dbuf_read_hole() and dropped the
BP_IS_HOLE() recheck that followed dnode_block_freed(), together with
the comment explaining it. That recheck was not about overrides: it
closes a race with dnode_sync().
dnode_sync_free_ranges() drops dn_mtx, clears the bps through
dnode_sync_free_range_impl() -> free_blocks(), then retakes dn_mtx and
removes the range from dn_free_ranges. For blkptrs held in the dnode
(dn_nlevels == 1), free_blocks() writes the bp without the parent's
db_rwlock, so holding that lock in dbuf_read() does not serialize
against it. A reader that saw a non-hole bp, then waited on dn_mtx in
dnode_block_freed() until the range was removed, gets "not freed" while
the bp it is about to copy is already a hole (with HOLE_BIRTH, lsize
and birth preserved). dbuf_read_impl() then hands that hole to
arc_read(): no child I/O is issued, checksum verification fails with
EINVAL, and without ZIO_FLAG_CANFAIL the pool is suspended.
[19 lines not shown]
Fix a deadlock between zvol first open and pool export
After the reference check, zpool export drops the namespace lock with
spa_export_thread still set, and then removes the pool's zvols. A zvol
first open can take the namespace lock in that window. It then waits in
spa_lookup() for the export to finish while it holds zv_state_lock.
Export needs that lock to remove the zvol, so neither can make progress.
zvol_first_open() now fails with ENXIO when spa_lookup() would wait for
the pool. That is the same error the open gets a moment later, once
export marks the zvol for removal. An open in that window can never
succeed, because export removes the zvol before anything else can fail.
So only the case that hangs today behaves differently.
The check is in the shared zvol_first_open(), so it covers Linux,
FreeBSD, and opens made while the caller already holds the namespace
lock, such as a pool that uses zvols from another pool as vdevs. The
new spa_lookup_would_wait() shares its code with spa_lookup().
[10 lines not shown]
zfs_file: hide implementing struct
Nothing cares about what zfs_file_t actually is, so we can remove it
from the public header and with it the platform gates.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19231
zfs_file: replace zfs_file_t in ztest to simulate failure
This is the only place that reaches into an existing zfs_file_t to mess
with its contents. Now that zfs_file_get() works on an existing (non-)
fd, we can just replace the zfs_file_t outright.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19231
zfs_file: implement _get/_put for userspace
Simple enough just to wrap the existing fd, whatever it is, in a
zfs_file_t.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19231
zfs: honour the refreserv alias in zfs create -V
When creating a volume without -s, the zfs command adds the default
reservation unless the property list already has one. It only looks
for the full property name, so a reservation given with the documented
"refreserv" alias is overridden by the default:
# zfs create -V 1G -o refreserv=none pool/vol
# zfs get -Hpo value refreservation pool/vol # 1G plus metadata
Look for the property in the validated list instead, where aliases
have been resolved to the full name.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19227
libzfs: restore the reservation when a volsize change fails
When the volsize of a volume changes, libzfs also sets the matching
reservation if the volume is thick provisioned, or if
refreservation=auto is given. The kernel sets the two properties
independently, so when the volsize cannot change but the reservation
can, e.g. on a read-only volume or one whose encryption key is not
loaded, the volume is left reserving space for a size it does not
have, and later volsize changes no longer adjust the reservation.
libzfs already puts the old volsize back when the reservation fails
with ENOSPC. Also put the old reservation back when the kernel's
error list has the volsize but not the reservation: set the old local
value again, or drop the new local value so that the default or
received one applies, as "zfs inherit -S" does.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19227
libzfs: support refreservation=auto in zfs_create()
zprop_parse_value() stores refreservation=auto as UINT64_MAX for
zfs_fix_auto_resv() to replace with the real size, which "zfs set" and
"zfs clone" do. zfs_create() passes it to the kernel as is, so
"zfs create -V 1G -o refreservation=auto pool/vol" fails with "out of
space", as does any libzfs consumer that asks for "auto". A plain
"zfs create -V" is unaffected only because the zfs command computes
the reservation itself.
Resolve "auto" in zfs_create() from the volsize and volblocksize being
created. The volume gets the reservation that "zfs create -V" sets
for the same volsize and volblocksize, so zfs_add_synthetic_resv()
keeps it in step with later volsize changes. As with any explicit
refreservation, -s does not override it.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19227
libzfs: compute the refreservation=auto value in one place
zfs_add_synthetic_resv() keeps a volume's reservation in step with its
volsize only while it equals zvol_volsize_to_reservation() for the
current volsize and volblocksize, and zfs_fix_auto_resv() sets that
same value when "zfs set" or "zfs clone" is given refreservation=auto.
Both build the same one-entry nvlist to get it. Move that into
zvol_auto_resv() so they share one computation, which zfs_create()
will use next.
No functional change.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19227
kmodtool: use kABI-baseline naming on RHEL to prevent kmod accumulation
On RHEL, errata kernels within a minor release share a stable kABI.
The existing kmodtool generates a unique kmod package name per exact
kernel version (e.g. kmod-zfs-5.14.0-687.52.1.el9_8), which causes
unbounded kmod package accumulation as new errata kernels are installed
and akmods rebuilds for each one.
Port the kABI-aware naming logic from RPM Fusion's kmodtool:
- Add init_kernel_uname_r_vars() to parse kernel uname -r into
components including the kABI baseline (kernel_uname_r_short).
Example: 5.14.0-687.52.1.el9_8.x86_64 -> 5.14.0-687.el9_8
- On RHEL (%{?rhel}), use kernel_uname_r_short in package names
so all errata kernels within a minor release produce the same
package name (e.g. kmod-zfs-5.14.0-687.el9_8).
- On Fedora (no kABI guarantee), keep the full kernel_uname_r in
[18 lines not shown]
vdev_raidz: keep unverified dRAID rebuild repairs speculative
A sequential rebuild has no checksum, so a dRAID row reconstructed with
all of its parity cannot be verified. Such repairs carry
ZIO_FLAG_SPECULATIVE, which limits a repair of a spare or replacing
column to the device being rebuilt. Since 0d1f3b1c6 skips parity
regeneration when no parity column failed, the flag is set only on the
remaining path, and a row whose missing column holds data repairs every
child of a spare or replacing column with unverified data, including
the original device.
Set the flag whenever a rebuild row is not verified by spare parity,
independently of regenerating parity.
Cover a rebuild onto a hot spare while another child is offline and
reads of the original child return flipped bits: the original must have
no checksum errors once the offline child returns and the pool is
scrubbed.
[3 lines not shown]
gettext: remove linking to libintl
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19219
config: remove gettext, iconv and related detection
Since we are no longer using libintl or gettext, we no longer need to
detect it. This lets us get rid of a lot of autoconf support that wasn't
used anywhere else. This is largely reverting e8864b1b28.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19219
gettext: remove calls to textdomain()
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19219
gettext: remove calls to dgettext()
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19219
gettext: remove calls to gettext()
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19219
gettext: remove libintl.h inclusion
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19219
Add alloc factor logic
RAIDz vdevs always perform allocations in multiples of nparity + 1
sectors. This is to prevent fragmentation from leaving small scattered
records everywhere that cannot be allocated and just pollute the
spacemaps. This rule was always preserved until the introduction of the
allocation range code, which allows allocation of not just specific
sizes, but anywhere in a range. The sizes that end up being allocated
may not be a multiple of nparity + 1 sectors in certain cases (primarily
when the allocation is at the end of a metaslab). If that happens, when
converting from the asize to the psize and back, we can end up with two
different psize values, which can theoretically cause issues in a few
different ways.
This PR adds logic to the metaslab code to check if there is a factor
that the allocations must be a multiple of, and if we're doing a
dynamically sized allocation, ensures that the allocation meets that
requirement.
[5 lines not shown]
ztest: do not use a detached vdev in online_vdev()
online_vdev() drops the config lock to online a device. If the
configuration changed meanwhile, the vdev may have been detached or
replaced and freed, which the function detects before returning. Its
verbose message then still read the old top-level vdev's state, a
use-after-free that AddressSanitizer reports during zloop runs, and
both abort paths returned the possibly freed vdev to stop the walk.
Log the saved GUID and expected generation together with the current
configuration generation, without dereferencing the old vdev. Stop
the walk with the root vdev, which stays valid under the reacquired
lock.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19224
docs: add repository notes for coding agents
Document portability, compatibility, safe testing and patch preparation
for coding agents. Share the instructions through a CLAUDE.md symlink.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #19230
ZTS: restore metaslab_force_ganging after removal_with_ganging
The cleanup of removal_with_ganging sets metaslab_force_ganging to
131073 instead of the value it replaced. Every later test on the same
runner then gangs about 3% of its blocks larger than 128K, since
metaslab_force_ganging_pct defaults to 3. A test that needs a plain
1M block fails intermittently; replacement/indirect_reconstruct did on
FreeBSD CI.
Save the tunable before changing it and restore it on exit, as the
other tests that force ganging do. Register cleanup after the save,
before pool setup and the tunable write, so setup failures restore it
too.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19207
ZTS: preserve failures when saving and restoring tunables
save_tunable hides a failed get_tunable behind echo, and
restore_tunable hides a failed set_tunable64 behind rm. A test can
therefore report successful cleanup while leaving its changed tunable
in place and discarding the saved value.
Check the tunable and saved-value reads, and return a failed setter's
status before deleting the saved value. A failed restore then retains
the value for a retry, and log_must reports the cleanup failure.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19207
libzfs: fix a double free of the btree leaf cache
Since commit e903655c50 ("zpool: Add zpool status -vv error ranges"),
libzfs_init() creates the process-wide btree leaf cache and
libzfs_fini() destroys it, once per call. A process that holds more
than one handle, as zed does with four, leaks a cache on every init
after the first and frees the same cache again on every fini after the
first, so zed aborts with "double free detected" when it stops. A
handle still in use after another handle's fini also allocates from
the freed cache when zpool_get_errlog() builds its range tree.
Count the handles and create the cache with the first and destroy it
with the last, as libzfs_core_init() and libzfs_core_fini() do for the
shared /dev/zfs descriptor.
The ZFS_SENDRECV_MAX_NVLIST error path of libzfs_init() now calls
libzfs_fini() instead of undoing a few steps by hand, which also stops
it from leaking the URI regex and the libzfs_core hold, and the two
earlier error paths free the regex as well.
[4 lines not shown]
CI: apt-get update before installing smatch dependencies (#19237)
The smatch job installs packages without refreshing the runner's
package lists. Since 2026-09-30 the lists in the runner image point
at a libdbi-perl security update that the mirrors no longer carry, so
the install fails with "404 Not Found" and every smatch run fails
before smatch is built. Run apt-get update first, as #18609 did for
the purge step of the other workflows.
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Build the pool stats config without the namespace lock
The spa_open_common() function built the full config with vdev stats
while it held the namespace lock. Every pool lookup in the system waits
on that lock, so polling stats on a pool with many vdevs slowed down
unrelated pool operations. The config is now built after the lock is
dropped. spa_config_generate() takes the config locks it needs. The
recovery load info is copied while the lock is still held.
A stats caller now holds the pool for as long as the build takes, which
can be hundreds of milliseconds on a pool with many vdevs. Export fails
with "pool is busy" when it sees such a hold, so running zpool iostat or
zpool status in a loop would break zpool export, zpool destroy and
zpool split. To avoid this, stats callers count their hold, and export
waits for that count to drop to zero before it sets spa_export_thread.
New stats callers wait while the pool is being exported, so the wait
always ends. The hold and its count are taken and dropped together
under the namespace lock. As before, export holds the namespace lock
from setting spa_export_thread until it checks the pool references.
[8 lines not shown]
Linux: don't fail send/recv on job control stop
Suspending a zfs send or receive that writes to or reads from a pipe
with ^Z kills it: "cannot send: signal received" for send, and
"internal error: Bad file descriptor" plus an abort for receive.
On Linux the stream is written in the ioctl thread itself. A pending
SIGTSTP makes a blocking pipe_write()/pipe_read() return -ERESTARTSYS
or a short count, and the send/recv path treats that as a fatal error.
issig() already knows how to stop the thread on SIGSTOP/SIGTSTP (commit
414f7249d), but it is only reached after the I/O error has been
recorded.
Retry the interrupted kernel_write()/kernel_read() in zfs_file_write()
and zfs_file_read() when issig() reports that the pending signal was
only a stop signal: the thread stops there, and the I/O continues from
where it stopped once it is resumed. Any other signal still aborts the
operation as before.
[3 lines not shown]
vdev_id: add sas_expander topology
Certain external JBODs present multiple internal expanders (front and
backside slots, for example), which each have respective bay and phy 0,
and so on. This results in none of the current topologies being able
to sensibly map such a setup. Disks will inevitably overlap, since
the same port has multiple disks in bay or phy 0.
This adds a new topology, sas_expander, which assigns disks to channels
based on the address of the rightmost expander in their path.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Timo Rothenpieler <timo at rothenpieler.org>
Closes #19070
Walk snapshot names without holding the objset in list-next ioctl
ZFS_IOC_SNAPSHOT_LIST_NEXT walks the snapnames ZAP, which lives in the
MOS, so it only needs the dataset. It held the objset instead, and for
an unmounted filesystem or an idle volume that means building a complete
objset_t and tearing it down again on every call, once per snapshot
returned.
Hold the pool and the dataset, and move the ZAP walk into
dsl_dataset_snapshot_list_next(), which dmu_snapshot_list_next() now
wraps for its remaining callers. Listing a dataset's snapshots no
longer fails when the dataset's own objset block cannot be read.
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Reviewed-by: Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19208
