Prevent range tree corruption race by updating dnode_sync()
Switch to incremental range tree processing in dnode_sync() to avoid
unsafe lock dropping during zfs_range_tree_walk(). This also ensures
the free ranges remain visible to dnode_block_freed() throughout the
sync process, preventing potential stale data reads.
This patch:
- Keeps the range tree attached during processing for visibility.
- Processes segments one-by-one by restarting from the tree head.
- Uses zfs_range_tree_clear() to safely handle ranges that may have
been modified while the lock was dropped.
- adds ASSERT()s to document that we don't expect dn_free_ranges
modification outside of sync context.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alek Pinchuk <apinchuk at axcient.com>
Issue #18186
Closes #18235
ci: update FreeBSD CI images from 14.3 to 14.4
Update FreeBSD CI targets from 14.3 to 14.4 in both the QEMU
start script and the workflow configuration.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18362
pyzfs: update license tags/classifiers
The standard for package license metadata[1] is a SPDX identifier in the
the `license` and that's all. So, updating that, remove the deprecated
license classifier, and adding a tag at the top of the file for
spdxcheck to find.
1. https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18356
zpool-list.8: clarify that only imported pools are listed (#18352)
The man page stated "all pools in the system are listed" which is
misleading, as only imported pools are shown. Clarify this and
add a cross-reference to zpool-import(8) for discovering pools
available for import.
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Linux 7.0: autoconf: Remove copy-from-user-inatomic API checks (#18348) (#18354)
This function was removed in c6442bd3b643: "Removing old code outside
of 4.18 kernsls", but fails at present on PowerPC builds due to the
recent inclusion of 6bc9c0a90522: "powerpc: fix KUAP warning in VMX
usercopy path" in the upstream kernel, which introduces a use of
cpu_feature_keys[], which is a GPL-only symbol. Removing the API
check as it doesn't appear necessary.
Signed-off-by: John Cabaj <john.cabaj at canonical.com>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
CI: Add ARM builder
Do a ZFS build inside of an ARM runner. This only does a simple
build, it does not run the test suite. The build runs on the
runner itself rather than in a VM, since nesting is not supported on
Github ARM runners.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #18343
CI: Support repository variable override for ZTS OS selection
Allow restricting ZTS OS targets by setting the vars.ZTS_OS_OVERRIDE
repository variable (e.g. '["debian13"]') to reduce shared runner
contention when running the full OS matrix is unnecessary. When unset,
the existing ci_type-based OS selection is used unchanged.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Ameer Hamza <ahamza at ixsystems.com>
Closes #18342
linux/super: flatten zpl_fill_super into zpl_get_tree
Target of opportunity; with no other callers, there's no need for it to
be a static function.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18339
linux/super: flatten zpl_mount_impl into zpl_get_tree
Target of opportunity; with no other callers, there's no need for it to
be a static function.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18339
linux/super: flatten mount/remount into get_tree/reconfigure
With the old API gone, there's no need to massage new-style calls into
its shape and call another function; we can just make those handlers
work directly.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18339
linux/super: remove support for old mount API
Removing the HAVE_FS_CONTEXT gates and anything that would be used if it
wasn't set.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18339
config: refuse to build without fs_context
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18339
zpool-iostat.8: clarify first report shows per-second averages
The first report printed by zpool iostat shows average per-second
values since boot, not cumulative totals. Clarify this in the man
page to avoid user confusion.
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18337
Linux 7.0: also set setlease handler on directories (#18331)
It turns out the kernel can also take directory leases, most notably in
the NFS server. Without a setlease handler on the directory file ops,
attempts to open a directory over NFS can fail with EINVAL.
Adding a directory setlease handler was missed in 168023b603. This fixes
that, allowing directories to be properly accessed over NFS.
Sponsored-by: TrueNAS
Reported-by: Satadru Pramanik <satadru at gmail.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
ZTS: Fix L2ARC test reliability
Disable depth cap (L2ARC_EXT_HEADROOM_PCT=0) in DWPD and multidev tests
that rely on predictable marker advancement during fill and measurement.
Rework multidev_throughput to verify sustained throughput across three
consecutive windows instead of asserting an absolute rate. Use larger
cache devices (8GB) to avoid frequent global marker resets
(smallest_capacity/8), fill ARC before attaching caches to provide a
stable evictable buffer pool, and lower write_max to 8MB/s to avoid
exhausting data within the measurement period.
Use destroy_pool (log_must_busy) instead of log_must zpool destroy
to handle transient EBUSY during teardown.
Remove l2arc_multidev_throughput_pos from the expected-fail list in
zts-report.py.in.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[2 lines not shown]
zarcsummary: add man page
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18330
zilstat: add AUTHORS section to man page
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18329
ZTS: Add back redundancy_draid_spare3 exception
Observed again in the CI. Put the maybe exception back in place
and reference a newly created issue for this sporadic failure.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18320
Make zpool status dedup table support raw bytes -p output
Check if -p flag is enabled, and if so print dedup table with raw
bytes. Restructure the logic in zutil_pool to check if -p flag is
enabled before printing either the bytes or raw numbers.
Calls to print the data for DDT now all use
zfs_nicenum_format(). Increased DDT histogram column buffers
to 32 bytes to prevent truncation when -p is enabled.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Adi Gollamudi <adigollamudi at gmail.com>
Closes #11626
Closes #17926
README: add FreeBSD 14.4-RELEASE alongside 15.0
14.4 was announced yesterday. Also link to part of
FreeBSD's security page.
https://www.freebsd.org/security/#sup
presents a fairly comprehensive table of supported releases.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <robn at despairlabs.com>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Graham Perrin <grahamperrin at gmail.com>
Closes #18304
ZTS: redundancy_draid_spare{1,3} exceptions
Update the redundancy_draid_spare1 exception to reference an issue
which describes the failure.
Remove the exception for the redundancy_draid_spare3 test. I have
not observed it in local testing. If it reproduces in the CI we
can create a new issue for it and put back the exception.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18308
ZVOL: Restrict cloning with different properties
While technically its not a problem to clone between ZVOLs with
different properties, it might create expectation of new properties
being applied during data move, while actually it won't happen.
For copies and checksum it may mean incorrect safety expectations.
For dedup, compression and special_small_blocks -- performance and
space usage.
This is a replica of #18180 from FS.
Reviewed-by: Ameer Hamza <ahamza at ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18315
ZVOL: Add encryption key check for block cloning
Somehow during block cloning porting from file systems was missed
the check for identical encryption keys. As result, blocks cloned
between unrelated ZVOLs produced authentication errors on later
reads. Having same or different encryption root does not matter.
This patch copies dmu_objset_crypto_key_equal() call from FS side.
Reviewed-by: Ameer Hamza <ahamza at ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18315
Fix the send --exclude option to work with encryption
When using --exclude, filtering needs to take place in two places:
in zfs_main.c via the callback previously added to support the
options, and in libzfs_sendrecv.c because it generates the nvlist
during a first pass, and that results in it complaining if the
excluded dataset is not available for sending. (eg, excluding an
encrypted dataset so you don't have to use --raw wouldn't work,
because the first pass would look at the dataset and decide you
couldn't use it.) Add send --exclude tests, including one that tests
excluding an encrypted hierarchy.
Reviewed-by: Allan Jude <allan at klarasystems.com>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Sean Eric Fagan <sef at kithrup.ie>
Closes #18278
zstream: add a drop_record subcommand
It can be used to drop extraneous records in a send stream caused by a
corrupt dataset, as in issue #18239.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alan Somers <asomers at gmail.com>
Sponsored by: ConnectWise
Closes #18275
libzfs: scrub: only include start and end nv pairs if needed for scrub
This patch addresses running `zpool scrub <pool>` with ZFS 2.4 userspace
while the loaded kernel module is still 2.3, failing with:
```
cannot scrub <pool>: the loaded zfs module does not support an option
for this operation. A reboot may be required to enable this option.
```
Checking for the source of the message via `strace` showed the scrub
ioctl failing and setting errno to ZFS_ERR_IOC_ARG_UNAVAIL[0]. With
that and the comments in `module/zfs/zfs_ioctl.c`[1] commit: 894edd084
seemed like a likely cause for the backward incompatibility.
The corresponding kernelspace code in `module/zfs/zfs_ioctl.c` defaults
to a setting of 0 if either parameter is not set, so not providing the
nvpairs in case both are 0 should not make a semantic difference.
Tested by:
[19 lines not shown]
Add the --file-layout (-f) option to zdb(8)
Displays the physical raidz block layout for a given file.
This leverages the internal vdev_raidz_map_alloc() function to find
the map of how the block data is laid out across the child disks.
The column entry for each row looks like:
+------------+
| D2 43 |
| 6020da |
+------------+
representing here the logical data column 2 that is 43 sectors high
starting at sector 0x6020da.
With -H, the output is a list of disks, LBAs, and block counts,
given in 512 byte block values.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
[4 lines not shown]
config: fix STATX_MNT_ID detection
statx(2) requires _GNU_SOURCE to be defined in order for sys/stat.h to
produce a definition for struct statx and the STATX_* defines. We get
that at compile time because we pass -D_GNU_SOURCE through to
everything, but in the configure check we aren't setting _GNU_SOURCE, so
we don't find STATX_MNT_ID, and so don't set HAVE_STATX_MNT_ID.
(This was fine before ccf5a8a6fc, because linux/stat.h does not require
_GNU_SOURCE).
Simple fix: in the check, define _GNU_SOURCE before including
sys/stat.h.
Sponsored-by: TrueNAS
Reviewed-by: Ameer Hamza <ahamza at ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18312
contrib/debian: add zilstat.1 manpage to installation list
Missed in 65165df129, which caused Debian packaging to fail.
Sponsored-by: TrueNAS
Reviewed-by: Ameer Hamza <ahamza at ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18313
zpool clear: remove undocumented rewind flags
Remove the -F, -n, and -X flags from zpool clear. These flags were
inherited from OpenSolaris but are not applicable in this context.
Unlike zpool import, where the pool is not yet loaded and a specific
TXG can be selected, zpool clear operates on an already imported pool
whose in-memory state is ahead of what is on disk. Rewinding
transactions would require force-exporting the pool first.
The rewind policy passed to zpool_clear() is now always
ZPOOL_NO_REWIND.
Tested on FreeBSD 16.0-CURRENT (amd64). Verified that -F, -n, and
-X are properly rejected as invalid options and that the usage output
reflects the change.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #13825
Closes #18300
