dsl_bookmark: fix redaction list refcount tag when upgrading spill
rl_bonus and rl_dbuf are expected to have the same hold tag if they are
different. If the spill hold is taken after the redaction_list_t was
created and the bonus hold was taken, it must also be taken with the
same tag. Fortunately, we have it right here, so we can just use it.
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18536
ddt_log: fix refcount tag between ddt_log_begin & ddt_log_commit
We have to hold and release the dbuf array with the same tag. Since the
caller provides the ddt_log_update_t and is managing its lifetime, and
the begin/commit calls must be matched, it's quite reasonable to its
pointer as the refcount tag.
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18536
zap: fix refcount tag use in zap_lookup_length_uint64 and zap_prefetch_uint64
The same tag must be used for zap_lockdir() and zap_unlockdir(), so we have
to follow the pattern used elsewhere: pass the tag used for
zap_lockdir() through to the _impl(), so it can use it for
zap_unlockdir().
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18536
linux: suppress reclaim lockdep in zfs_inactive via rwlock wrappers
kswapd can enter zfs_inactive() from inode reclaim while holding
fs_reclaim. The z_teardown_inactive_lock still serializes teardown,
but the reclaim-thread acquire/release pair can produce a lockdep
cycle through zfs_zinactive() and zfs_rmnode().
Add Linux rwlock nolockdep wrappers alongside the existing rwlock
macros and use them only for the reclaim-thread
z_teardown_inactive_lock acquire/release in zfs_inactive(). Keep
the real rwsem semantics unchanged and leave CONFIG_LOCKDEP
handling in the platform rwlock layer.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ZhengYuan Huang <gality369 at gmail.com>
Closes #18505
CI: Fix 99.99 META version
We have an option in zfs-qemu-packages to test against a specific kernel
version. However, qemu-3-deps.sh was incorrectly hard coded to look
at $2 for a kernel version argument (which could come in $2 or $3
depending on if --poweroff was also passed). This caused the CI
to incorrectly edit META with a max supported kernel version of 99.99
when we didn't want that.
Fix this by looking at all the arguments for something that looks
like a kernel version and set that as the kernel max in META.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #18526
Closes #18531
Remove arc_bcopy_func() function
While this function could be convenient it appears it's never been
used. In practice, callers end up using the arc_getbuf_func()
instead. Remove this unused function.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18534
arc: export additional required symbols
External consumers of arc_read() need to be able to destroy the
returned arc_buf_t. Add the arc_buf_destroy() interface as an
exported symbol.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18533
ZTS: zhack_metaslab_leak.ksh busy export
If the pool is active 'zpool export' will fail resulting in
a test failure. Swap log_must with log_must_busy so the export
is retried when reported as busy before failing the test.
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18512
Integrate DDT and BRT tests
Don't disable block cloning during dedup tests. Just don't use
cp to not trigger it. Add a new test, explicitly mixing dedup
and cloning on the same file, that should be handled by DDT.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18520
Fix double free for blocks cloned after DDT prune
Before this change, for blocks marked with D flag but absent in DDT
(pruned from it), zio_ddt_free() fell back to ZIO_STAGE_DVA_FREE
without trying ZIO_STAGE_BRT_FREE first. Same time such blocks
might be present in BRT, and not handling that would result in
double/multiple free.
This change makes ZIO_DDT_FREE_PIPELINE include ZIO_FREE_PIPELINE,
just adding required ZIO_STAGE_ISSUE_ASYNC and ZIO_STAGE_DDT_FREE,
and moves DDT stages before BRT. This way, if the block is found
in DDT by zio_ddt_free(), the pipeline is short-circuited to
ZIO_INTERLOCK_PIPELINE, similar to what zio_brt_free() does. If
not, then BRT is checked, and if also no match, the block is freed.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18520
linux/zpl_super: handle 'source' option directly
vfs_parse_fs_param_source() didn't appear until 5.14, and was not
backported to kernel.org LTS kernels. It's simple enough that it's
easier to just handle it ourselves rather than use a configure check.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18529
Linux: avoid znode list lock inversion during resume
Lockdep reports a circular locking dependency during mounted filesystem
rollback. zfs_resume_fs() walks z_all_znodes under z_znodes_lock and
calls zfs_rezget(), which takes the per-object znode hold lock via
zfs_znode_hold_enter().
The normal zget path takes these locks in the opposite order.
zfs_zget() takes the per-object hold lock before zfs_znode_alloc()
inserts the znode on z_all_znodes under z_znodes_lock. Resume can
therefore establish z_znodes_lock -> zh_lock while normal lookup
creates zh_lock -> z_znodes_lock.
Pin the current and next znodes with igrab() while holding the list
lock, then drop the list lock before reloading the znode. Existing
stale inode handling is preserved, and both the suspended reference
and temporary walk reference are released asynchronously.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ZhengYuan Huang <gality369 at gmail.com>
Closes #18517
Linux: expose zfs_arc_no_grow_shift as a module parameter
The zfs_arc_no_grow_shift variable is tunable via sysctl on FreeBSD
but had no module parameter registration on Linux.
Register it once in arc.c using param_get_uint and a per-platform
set handler, replacing the FreeBSD-only registration.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alek Pinchuk <alek.pinchuk at connectwise.com>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18461
CI: FreeBSD 15.1 STABLE
Update the freebsd15-1s builder to the released STABLE image.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18524
libzfs_pool: document export and initialize functions
Add brief docstrings to zpool_export(), zpool_export_force(),
zpool_initialize() and zpool_initialize_wait().
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18514
Consistently encode DRR_BEGIN packed nvlist payloads with NV_ENCODE_XDR
Currently, zfs send generates a mix of nvlist encodings in DRR_BEGIN
records, some XDR and some in native byte order. The result is that
most streams currently can't be zfs received on opposite-endian systems.
zfs send generates the outer wrappers for compound streams in userspace,
and it explicitly requests NV_ENCODE_XDR format for those records. But
the BEGIN records for individual datasets are generated on the kernel
side, in dmu_send.c, where fnvlist_pack() is used for encoding. That
routine hard-wires NV_ENCODE_NATIVE format.
This PR replaces the fnvlist_pack() call with a direct call to
nvlist_pack() that specifies NV_ENCODE_XDR.
Tests are included to verify that native-encoded nvlists are not
generated by any kernel path that attaches nvlists to BEGIN records.
There's also a check for XDR encoding in the outer wrapper of
replication streams in case there is ever a regression there.
[15 lines not shown]
zap: internal interface cleanup
Similar to previous, though a much lighter touch because these are not
"public" interfaces.
- reorganising functions into groups, by rough function class.
- matching header order to source order, to make it a little easier to
find things.
- adding light documentation to functions that had none.
Note that I've not added any documentation for the mzap_* and fzap_*
functions, as part of this commit series is laying the groundwork to
hide those functions in their backend modules; such documentation would
become obsolete very quickly.
No actual code changes.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[4 lines not shown]
zap: public interface cleanup
- reorganising functions into groups, collections of the variants of the
same function.
- matching header order to source order, to make it a little easier to
find things.
- moving per-function documentation from source to header.
- adding light documentation to functions that had none.
No actual code changes.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Akash B <akash-b at hpe.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18516
zap: split implementation out into more files
The ZAP code is mixed up across a few files without clear separation of
concerns. This splits it out from three source files to five:
- zap.c: the bulk of the "public" interface
- zap_impl.c: internals shared across all backends
- zap_micro.c: microzap backend
- zap_fat.c: fatzap backend: core logic
- zap_leaf.c: fatzap backend: leaf blocks
Note that this doesn't not change any code, just moves functions around.
Also note that right now the microzap and fatzap backends know more
about each other than is healthy. This change is simply marking out
where different things should live in the end, to make it easier for
that refactoring work to begin.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[4 lines not shown]
dsl_dir: avoid dd_lock during snapshots_changed updates
Avoid holding dd_lock while updating the on-disk
snapshots_changed timestamp.
Both dsl_dir_zapify() and zap_update() may dirty buffers
and recurse into space accounting, which can take dd_lock.
Holding dd_lock across either operation can therefore
preserve the lock-order inversion reported by lockdep.
Only protect the in-memory dd_snap_cmtime update
with dd_lock. Perform the zapify and ZAP update without
dd_lock held, and retry the on-disk write if another updater
advanced dd_snap_cmtime while the write was in progress.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ZhengYuan Huang <gality369 at gmail.com>
Co-authored-by: gality369 <gality369 at example.com>
Closes #18472
zpool-attach.8: add EXAMPLES section
Mirror-attach (shared with zpool.8 example 5) and raidz expansion.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18508
CI: skip qemu matrix for documentation-only pull requests
Add a new "docs" CI type, selected when every file modified by a
pull request matches a documentation pattern (man pages, .md,
AUTHORS, COPYRIGHT, LICENSE, NOTICE, .gitignore). For this type the
os_selection is empty and the qemu matrix runs no jobs.
This affects only pull requests whose entire diff is documentation.
Any change touching a non-documentation file continues to be
classified as full, quick, linux, or freebsd by the existing
file-path rules, and a manual ZFS-CI-Type commit tag still overrides
that classification.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18518
zfs.4: Fix documentation of zfs_arc_dnode_reduce_percent
Fixes: 25458cbef Limit the amount of dnode metadata in the ARC
Fixes: 5b9f3b766 Soften pruning threshold on not evictable metadata
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Mateusz Piotrowski <0mp at FreeBSD.org>
Closes #18513
initramfs: fix incorrect variable rename
Fixes regression introduced by 61ab032ae0391bce38aef1e43b5b930724ecdb55.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Co-Authored-By: Claude Sonnet 4.6 <noreply at anthropic.com>
Signed-off-by: Joel Low <joel at joelsplace.sg>
Closes #18442
Linux: annotate nested xattr setattr znode locks
zfs_setattr() updates both the target znode and its hidden xattr
directory when ownership, mode, or project ID changes. The xattr
directory uses the same z_acl_lock and z_lock classes as the
parent znode, so lockdep reports recursive locking when the
second znode's mutexes are acquired.
This is a lockdep false positive rather than a real deadlock.
attrzp is the target file's hidden xattr directory, and the code
does not acquire these znode mutexes in the reverse order.
Acquire the attrzp mutexes with mutex_enter_nested() so lockdep
treats them as nested.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ZhengYuan Huang <gality369 at gmail.com>
Co-authored-by: gality369 <gality369 at example.com>
Closes #18506
zarcstat: detect attached L2ARC device with no data
zarcstat and zarcsummary detected L2ARC presence using the l2_size
kstat, which is data held in L2ARC, not whether a cache device is
attached. When a cache device was attached but empty (freshly added,
or fully evicted):
- zarcstat rejected "-f l2*" with "Incompatible field specified!"
- zarcsummary printed "L2ARC not detected, skipping section",
hiding cumulative I/O history and health counters
Expose the existing l2arc_ndev counter as a new kstat l2_dev_count.
It is maintained by l2arc_add_vdev() and l2arc_remove_vdev(), so it
tracks attachment in real time. Use it in both tools, falling back to
l2_size for compatibility with older kernel modules.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Ameer Hamza <ahamza at ixsystems.com>
Closes #18499
zdb: detect BRT and DDT leaks during block traversal
During -b traversal, track BRT and DDT reference counts and report
blocks claimed more times than their reference tables account for
if it causes claim errors, instead of just asserting it. Also
report entries with references not fully consumed by the traversal.
Add zdb leaks checks to cloning and dedup tests. This should make
sure the pools are in a sane state after completing the functional
tests.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18494
ZTS: redundancy_draid_spare1
Preserve the 'zpool status' output used to calculate the number of
checksum errors so it can be logged on failure. Several instances have
been observed in the CI where cksum was set to a non-zero value, yet a
subsequent run of 'zpool status' on failure showed no checksum errors.
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18500
