OpenZFS/src 54126fdman/man4 zfs.4, module/os/linux/zfs arc_os.c

set zfs_arc_shrinker_limit to 0 by default

zfs_arc_shrinker_limit was introduced to avoid ARC collapse due to
aggressive kernel reclaim. While useful, the current default (10000) is
too prone to OOM especially when MGLRU-enabled kernels with default
min_ttl_ms are used. Even when no OOM happens, it often causes too much
swap usage.

This patch sets zfs_arc_shrinker_limit=0 to not ignore kernel reclaim
requests. ARC now plays better with both kernel shrinker and pagecache
but, should ARC collapse happen again, MGLRU behavior can be tuned or
even disabled.

Anyway, zfs should not cause OOM when ARC can be released.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Gionatan Danti <g.danti at assyoma.it>
Closes #16909
DeltaFile
+2-2man/man4/zfs.4
+2-2module/os/linux/zfs/arc_os.c
+4-42 files

OpenZFS/src 9dd5fe1include/sys zvol_impl.h, module/zfs zvol.c zfs_vnops.c

zvol: implement platform-independent part of block cloning

In Linux, block devices currently lack support for `copy_file_range`
API because the kernel does not provide the necessary functionality.
However, there is an ongoing upstream effort to address this
limitation: https://patchwork.kernel.org/project/dm-devel/cover/20240520102033.9361-1-nj.shetty@samsung.com/.
We have adopted this upstream kernel patch into the TrueNAS kernel and
made some additional modifications to enable block cloning specifically
for the zvol block device. This patch implements the platform-
independent portions of these changes for inclusion in OpenZFS.
This patch does not introduce any new functionality directly into
OpenZFS. The `TX_CLONE_RANGE` replay capability is only relevant when
zvols are migrated to non-TrueNAS systems that support Clone Range
replay in the ZIL.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Ameer Hamza <ahamza at ixsystems.com>
Closes #16901 
DeltaFile
+283-1module/zfs/zvol.c
+5-0include/sys/zvol_impl.h
+1-1module/zfs/zfs_vnops.c
+289-23 files

OpenZFS/src a153397tests/zfs-tests/tests/functional/redacted_send redacted_panic.ksh

ZTS: Reduce file size in redacted_panic to 1GB

This test takes 3 minutes on RELEASE FreeBSD bots, but on CURRENT,
probably due to debugging it has in kernel, it does not complete
within 10 minutes, ending up killed.  As I see all the redacting
here happens within the first ~128MB of the file, so I hope it
won't matter if there is 1GB of data instead of 2GB.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Signed-off-by:Alexander Motin <mav at FreeBSD.org>
Sponsored by: iXsystems, Inc.
Closes #11141
DeltaFile
+1-1tests/zfs-tests/tests/functional/redacted_send/redacted_panic.ksh
+1-11 files

OpenZFS/src b66d910tests/zfs-tests/tests/functional/cli_root/zpool_import zpool_import_status.ksh

ZTS: Remove procfs use from zpool_import_status

procfs might be not mounted on FreeBSD.  Plus checking for specific
PID might be not exactly reliable.  Check for empty list of jobs
instead.

Premature loop exit can result in failed test and failed cleanup,
failing also some following tests.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Signed-off-by:Alexander Motin <mav at FreeBSD.org>
Sponsored by: iXsystems, Inc.
Closes #11141
DeltaFile
+1-6tests/zfs-tests/tests/functional/cli_root/zpool_import/zpool_import_status.ksh
+1-61 files

OpenZFS/src 8bf1e83tests/zfs-tests/include blkdev.shlib

ZTS: Remove non-standard awk hex numbers usage

FreeBSD recently removed non-standard hex numbers support from awk.
Neither it supports -n argument, enabling it in gawk.  Instead of
depending on those rewrite list_file_blocks() fuction to handle the
hex math in shell instead of awk.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Signed-off-by:Alexander Motin <mav at FreeBSD.org>
Sponsored by: iXsystems, Inc.
Closes #11141
DeltaFile
+4-15tests/zfs-tests/include/blkdev.shlib
+4-151 files

OpenZFS/src c4e5fa5tests/runfiles common.run, tests/zfs-tests/tests Makefile.am

ZTS: test clearing pool and vdev userprops

Confirming that clearing pool and vdev userprops produce the same
result: an empty value, with default source.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #16887
DeltaFile
+44-0tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_clear_userprop.ksh
+38-2tests/zfs-tests/tests/functional/cli_root/zpool_set/zpool_set_common.kshlib
+2-1tests/runfiles/common.run
+1-0tests/zfs-tests/tests/Makefile.am
+85-34 files

OpenZFS/src 03b7cfdmodule/zfs spa.c

spa_sync_props: remove pool userprops by setting empty-string

People have noted there's no way to remove a pool userprop, only zero
it. Turns vdev userprops had a method, by setting empty-string. So this
makes pool userprops follow the same behaviour.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #16887
DeltaFile
+11-3module/zfs/spa.c
+11-31 files

OpenZFS/src 779c5a5lib/libzfs libzfs_pool.c

zpool_get_vdev_prop_value: show missing vdev userprops

If a vdev userprop is not found, present it as value '-', default
source, so it matches the output from pool userprops.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #16887
DeltaFile
+2-1lib/libzfs/libzfs_pool.c
+2-11 files

OpenZFS/src 5beeabeman/man8 zfs-create.8

zfs-create(8): ZFS for swap: caution, clarity

Make the section heading more generic (the section relates to ZFS files
as well as ZFS volumes).

Swapping to a ZFS volume is prone to deadlock. Remove the related
instruction, direct readers to OpenZFS FAQ. Related, but not linked
from within the manual page:

<https://openzfs.github.io/openzfs-docs/Project%20and%20Community/FAQ.html#using-a-zvol-for-a-swap-device-on-linux>
(Using a zvol for a swap device on Linux).

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Graham Perrin <grahamperrin at freebsd.org>
Issue #7734
Closes #14756
DeltaFile
+5-8man/man8/zfs-create.8
+5-81 files

OpenZFS/src 89f796dtests/zfs-tests/tests/functional/raidz raidz_expand_001_pos.ksh raidz_expand_002_pos.ksh, tests/zfs-tests/tests/functional/redundancy redundancy_draid.ksh redundancy_draid_damaged1.ksh

ZTS: Increase write sizes for RAIDZ/dRAID tests

Many RAIDZ/dRAID tests filled files doing millions of 100 or even
10 byte writes.  It makes very little sense since we are not
micro-benchmarking syscalls or VFS layer here, while before the
blocks reach the vdev layer absolute majority of the small writes
will be aggregated.  In some cases I see we spend almost as much
time creating the test files as actually running the tests.  And
sometimes the tests even time out after that.

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by:  Alexander Motin <mav at FreeBSD.org>
Sponsored by:   iXsystems, Inc.
Closes #16905
DeltaFile
+3-3tests/zfs-tests/tests/functional/raidz/raidz_expand_001_pos.ksh
+3-3tests/zfs-tests/tests/functional/raidz/raidz_expand_002_pos.ksh
+3-3tests/zfs-tests/tests/functional/redundancy/redundancy_draid.ksh
+3-3tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged1.ksh
+3-3tests/zfs-tests/tests/functional/redundancy/redundancy_draid_damaged2.ksh
+3-3tests/zfs-tests/tests/functional/redundancy/redundancy_raidz.ksh
+18-184 files not shown
+25-2510 files

OpenZFS/src c37a2ddmodule/zfs zap_micro.c

microzap: set hard upper limit of 1M

The count of chunks in a microzap block is stored as an uint16_t
(mze_chunkid). Each chunk is 64 bytes, and the first is used to store a
header, so there are 32767 usable chunks, which is just under 2M. 1M is
the largest power-2-rounded block size under 2M, so we must set the
limit there.

If it goes higher, the loop in mzap_addent can overflow and fall into
the PANIC case.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #16888
DeltaFile
+15-3module/zfs/zap_micro.c
+15-31 files

OpenZFS/src 1acd246module/zfs vdev.c

Fix readonly check for vdev user properties

VDEV_PROP_USERPROP is equal do VDEV_PROP_INVAL and so is not a real
property.  That's why vdev_prop_readonly() does not work right for
it.  In particular it may declare all vdev user properties readonly
on FreeBSD.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <robn at despairlabs.com>
Signed-off-by:  Alexander Motin <mav at FreeBSD.org>
Sponsored by:   iXsystems, Inc.
Closes #16890
DeltaFile
+1-1module/zfs/vdev.c
+1-11 files

OpenZFS/src 219a89clib/libzfs libzfs_changelist.c

Skip iterating over snapshots for share properties

Setting sharenfs and sharesmb properties on a dataset can become costly
if there are large number of snapshots, since setting the share
properties iterates over all snapshots present for a dataset. If it is
the root dataset for which we are trying to set the share property,
snapshots for all child datasets and their children will also be
iterated.

There is no need to iterate over snapshots for share properties
because we do not allow share properties or any other property,
to be set on a snapshot itself execpt for user properties.

This commit skips iterating over snapshots for share properties,
instead iterate over all child dataset and their children for share
properties.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Umer Saleem <usaleem at ixsystems.com>
Closes #16877
DeltaFile
+14-2lib/libzfs/libzfs_changelist.c
+14-21 files

OpenZFS/src f00a57acmd/zfs zfs_main.c

zfs_main: fix alignment on props usage output

I guess we've got some long property names since this was first set up!

Sponsored-by: https://despairlabs.com/sponsor/
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Rob Norris <robn at despairlabs.com>
Closes #16883
DeltaFile
+16-16cmd/zfs/zfs_main.c
+16-161 files

OpenZFS/src e5ac778.github/workflows/scripts qemu-2-start.sh

CI: Fix FreeBSD 13.4 STABLE build

In #16869 we added FreeBSD 13.4 STABLE, but forget the special
thing, that the virtio nic within FreeBSD 13.x is buggy.

This fix adds the needed rtl8139 nic to the VM.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by:  Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Tino Reichardt <milky-zfs at mcmilk.de>
Closes #16885
DeltaFile
+1-0.github/workflows/scripts/qemu-2-start.sh
+1-01 files

OpenZFS/src ff7d051include/os/linux/spl/sys taskq.h, include/os/linux/zfs/sys abd_os.h

Fix compile-time warnings caused by duplicate struct typedefs (#16880)

Some compiler/versions warn these typedefs according to #16660.

The platform specific header sys/abd_os.h shouldn't define or use abd_t,
as it's defined in its non-platform specific consumer sys/abd.h.
Do the same as what FreeBSD header does.

Original-patch-by: Tomohiro Kusumi <kusumi.tomohiro at gmail.com>
Signed-off-by: Rob Norris <robn at despairlabs.com>
(cherry picked from commit a9851ea3dd6af6f789e221f2ccd7ad43561eff81)

Co-authored-by: Tomohiro Kusumi <kusumi.tomohiro at gmail.com>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
DeltaFile
+5-5include/os/linux/zfs/sys/abd_os.h
+1-0include/os/linux/spl/sys/taskq.h
+6-52 files

OpenZFS/src ab7cbbemodule/zcommon zpool_prop.c

zprop: fix value help for ZPOOL_PROP_CAPACITY

It's a percentage and documented as such, but we were showing it as
<size>.

Sponsored-by: https://despairlabs.com/sponsor/
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Rob Norris <robn at despairlabs.com>
Closes #16881 
DeltaFile
+1-1module/zcommon/zpool_prop.c
+1-11 files

OpenZFS/src 830a531.github/workflows zfs-qemu.yml, .github/workflows/scripts qemu-2-start.sh

CI: Add FreeBSD 14.2 RELEASE+STABLE builds

Update the CI to include FreeBSD 14.2 as a regularly tested platform.

Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #16869
DeltaFile
+10-10.github/workflows/scripts/qemu-2-start.sh
+5-4.github/workflows/zfs-qemu.yml
+15-142 files

OpenZFS/src 5c51f8fconfig kernel-kthread.m4, module/os/linux/spl spl-thread.c

config: fix dequeue_signal check for kernels <4.20

Before 4.20, kernel_siginfo_t was just called siginfo_t. This was
causing the kthread_dequeue_signal_3arg_task check, which uses
kernel_siginfo_t, to fail on older kernels.

In d6b8c17f1, we started checking for the "new" three-arg
dequeue_signal() by testing for the "old" version. Because that test is
explicitly using kernel_siginfo_t, it would fail, leading to the build
trying to use the new three-arg version, which would then not comile.

This commit fixes that by avoiding checking for the old 3-arg
dequeue_signal entirely. Instead, we check for the new one, as well as
the 4-arg form, and we use the old form as a fallback. This way, we
never have to test for it explicitly, and once we're building
HAVE_SIGINFO will make sure we get the right kernel_siginfo_t for it, so
everything works out nice.

Original-patch-by: Finix <yancw at info2soft.com>
Signed-off-by: Rob Norris <robn at despairlabs.com>
DeltaFile
+28-21config/kernel-kthread.m4
+3-3module/os/linux/spl/spl-thread.c
+31-242 files

OpenZFS/src 882a809config kernel-vfs-iov_iter.m4 kernel-pin-user-pages.m4, module/os/linux/zfs zfs_uio.c

Use pin_user_pages API for Direct I/O requests

As of kernel v5.8, pin_user_pages* interfaced were introduced. These
interfaces use the FOLL_PIN flag. This is preferred interface now for
Direct I/O requests in the kernel. The reasoning for using this new
interface for Direct I/O requests is explained in the kernel
documenetation:
Documentation/core-api/pin_user_pages.rst

If pin_user_pages_unlocked is available, the all Direct I/O requests
will use this new API to stay uptodate with the kernel API requirements.

Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Brian Atkinson <batkinson at lanl.gov>
Closes #16856
DeltaFile
+94-17module/os/linux/zfs/zfs_uio.c
+21-26config/kernel-vfs-iov_iter.m4
+33-0config/kernel-pin-user-pages.m4
+2-0config/kernel.m4
+150-434 files

OpenZFS/src c6442bdconfig kernel-vfs-direct_IO.m4 kernel-vfs-iov_iter.m4, include/os/linux/spl/sys uio.h

Removing old code outside of 4.18 kernsls

There were checks still in place to verify we could completely use
iov_iter's on the Linux side. All interfaces are available as of kernel
4.18, so there is no reason to check whether we should use that
interface at this point. This PR completely removes the UIO_USERSPACE
type. It also removes the check for the direct_IO interface checks.

Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Brian Atkinson <batkinson at lanl.gov>
Closes #16856
DeltaFile
+11-169module/os/linux/zfs/zfs_uio.c
+0-57config/kernel-vfs-direct_IO.m4
+7-45module/os/linux/zfs/zpl_file.c
+2-37config/kernel-vfs-iov_iter.m4
+5-12include/os/linux/spl/sys/uio.h
+1-2lib/libspl/include/sys/uio.h
+26-3221 files not shown
+26-3247 files

OpenZFS/src acda137module/zcommon simd_stat.c

simd_stat: fix undefined CONFIG_KERNEL_MODE_NEON error on armel

CONFIG_KERNEL_MODE_NEON depends on CONFIG_NEON. Neither is defined
on armel. Add a guard to avoid compilation errors.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Shengqi Chen <harry-chen at outlook.com>
Closes #16871
DeltaFile
+2-0module/zcommon/simd_stat.c
+2-01 files

OpenZFS/src 22259fbconfig kernel-xattr-handler.m4

Fix stray "no" in configure output

This is purely a cosmetic fix which removes a stray "no" from
the configure output.

Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Reviewed-by:  Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #16867
DeltaFile
+0-1config/kernel-xattr-handler.m4
+0-11 files

OpenZFS/src ff6266emodule/zfs vdev_raidz.c

Fix use-afer-free regression in RAIDZ expansion

We should not dereference rra after the last zio_nowait() is called.
It seems very unlikely, but ASAN in ztest managed to catch it.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by:  Alexander Motin <mav at FreeBSD.org>
Sponsored by:   iXsystems, Inc.
Closes #16868
DeltaFile
+2-2module/zfs/vdev_raidz.c
+2-21 files

OpenZFS/src 586304aetc/systemd/system zfs-mount.service.in

Remount datasets on soft-reboot

The one-shot zfs-mount.service is incorrectly deemed active by 
Systemd after a systemctl soft-reboot. As such, soft-rebooting
prevents zfs mount -a from being ran automatically.

This commit makes it so that zfs-mount.service is marked as being 
undone by the time umount.target is reached, so that zfs.target then 
pulls it in again and gets it restarted after a soft reboot.

Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: kotauskas <v.toncharov at gmail.com>
Closes #16845
DeltaFile
+7-0etc/systemd/system/zfs-mount.service.in
+7-01 files

OpenZFS/src 46e06femodule/os/freebsd/zfs vdev_geom.c, module/os/linux/zfs vdev_disk.c

flush: only detect lack of flush support in one place

It seems there's no good reason for vdev_disk & vdev_geom to explicitly
detect no support for flush and set vdev_nowritecache.  Instead, just
signal it by setting the error to ENOTSUP, and let zio_vdev_io_assess()
take care of it in one place.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #16855
DeltaFile
+0-15module/os/freebsd/zfs/vdev_geom.c
+3-3module/zfs/zio.c
+2-3module/os/linux/zfs/vdev_disk.c
+5-213 files

OpenZFS/src fbea924module/zfs zio.c

flush: don't report flush error when disabling flush support

The first time a device returns ENOTSUP in repsonse to a flush request,
we set vdev_nowritecache so we don't issue flushes in the future and
instead just pretend the succeeded. However, we still return an error
for the initial flush, even though we just decided such errors are
meaningless!

So, when setting vdev_nowritecache in response to a flush error, also
reset the error code to assume success.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #16855
DeltaFile
+5-2module/zfs/zio.c
+5-21 files

OpenZFS/src 76f57abconfig zfs-build.m4

build: use correct bashcompletiondir on arch

Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: poscat <poscat at poscat.moe>
Closes #16861 
DeltaFile
+1-1config/zfs-build.m4
+1-11 files

OpenZFS/src ecc0970lib/libspl backtrace.c

backtrace: fix off-by-one on string output

sizeof("foo") includes the trailing null byte, so all the output had
nulls through it. Most terminals quietly ignore it, but it makes some
tools misdetect file types and other annoyances.

Easy fix: subtract 1.

Sponsored-by: https://despairlabs.com/sponsor/
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <robn at despairlabs.com>
Closes #16862 
DeltaFile
+1-1lib/libspl/backtrace.c
+1-11 files

OpenZFS/src 7cbe7bb. META

Tag 2.3.0-rc4

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
DeltaFile
+1-1META
+1-11 files