deb: Use release number generated by ZFS for native builds
This allows for a dynamic release part of the version string as is done
for rpm and non-native deb package builds. Importantly, for git builds
this allows having the git commit in the package version string.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Glenn Washburn <development at efficientek.com>
Closes #19186
zpool-prefetch: Fix document description typo
Verb should be imperative mood matching rest of manual.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Ziaee <ziaee at FreeBSD.org>
Closes #19184
zfs_domount: fix vfs_t double-free on root setup failure
When zfs_root() or d_make_root() fails, zfs_domount() calls
zfs_umount(), which frees zfsvfs->z_vfs via zfsvfs_free(). After
the vfs_t lifetime was matched to fs_context, that vfs_t is still
owned by the caller in fc->fs_private. zpl_get_tree() then returns
without clearing fs_private, and put_fs_context() frees it again.
Detach z_vfs before zfs_umount() so the caller retains ownership,
matching the other zfs_domount() error paths.
This is a follow-up to the vfs_t lifetime change in #18377.
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Wale Zhang <wale.zhang.ftd at gmail.com>
Closes #19167
Linux: read the snapshot creation time from its bonus buffer
When a '.zfs/snapshot/<name>' entry is looked up by name (stat, open, or
a path walk through it; listing the directory does not do this),
zfsctl_inode_lookup() reads the snapshot's creation time for the
entry's btime through dsl_dataset_hold_obj(). That instantiates the
whole in-core dataset (dsl_dir hold, deadlists, neighbour references,
fsid uniqueness) and tears it down again on release, and it accounts
for most of a dentry-cold lookup. The value is a field of the
snapshot's dsl_dataset_phys_t, so read it from the bonus buffer the way
the fsid is read, through a helper both callers share.
Same value, same locking, same fallback on error. A warm lookup is
unchanged; a cold one loses the dataset instantiation, roughly an order
of magnitude on the systems it was measured on.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19174
Linux: report a snapshot's fsid from its .zfs/snapshot entry
statfs() on a control directory inode ('.zfs', '.zfs/snapshot', or an
unmounted snapshot entry reached without the automount) fails with EIO
because zfs_statvfs() verifies the znode's SA handle, which these inodes
lack. Handle them explicitly: '.zfs' and '.zfs/snapshot' report the
containing filesystem, and a '.zfs/snapshot/<name>' entry reports the
fsid of its snapshot, read from the dataset's bonus buffer (the objset id
is encoded in the entry's inode number) without instantiating the
dataset, so a probe costs a few microseconds.
This identifies a snapshot without mounting it: open the entry with
O_PATH, which does not trigger the automount, and fstatfs() it. The
value is what the snapshot's superblock reports once mounted, and path
based statfs() still automounts as before. An NFS server resolving a
file handle for an unmounted snapshot (expired, or imported on another
node) can probe entries instead of mounting every snapshot until one
matches. Add the statfs_nomount helper, the snapdir_statfs_fsid test
and a zfsconcepts(7) note.
[5 lines not shown]
zpool: accept more redundant special and dedup vdevs
The replication check required a special or dedup vdev to tolerate
exactly as many device failures as the normal vdevs in the pool. A
3-way mirror special vdev on a raidz1 pool, or on a 2-way mirror pool,
was rejected as a mismatch, and the only way past it was -f, which
also overrides unrelated checks.
Accept a special or dedup vdev which tolerates at least as many
failures as the normal vdevs of a redundant pool. Special and dedup
vdevs are now compared with the normal vdevs rather than with whichever
vdev happens to precede them, so normal vdevs are still required to
match each other. Less redundant special or dedup vdevs, and redundant
ones added to a non-redundant pool, are still rejected.
As a side effect, a pool that already has a more redundant special
vdev is now considered consistent, so later additions to it are
checked instead of being skipped.
[4 lines not shown]
pkg: Check for Berkeley DB RPM database format
Commit 0d5bb1d010 added a check to silence warnings when building
on Debian systems where the /var/lib/rpm/rpmdb.sqlite RPM database
does not exist. However, RHEL8 still uses the older Berkeley DB
RPM database format, not SQLite, and the primary database file is
instead named /var/lib/rpm/Packages. Setting an the alternate,
empty RPM_DBPATH results in the package build failing due to missing
build dependencies. Resolve the build failure by additionally
checking for the legacy primary database name.
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Glenn Washburn <development at efficientek.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #19181
CI: Remove centos-streamX from default builders
In practice, we've seen limited benefit from always running the
centos-stream9 and centos-stream10 builders as part of the CI.
Remove them from the default set of builders to reduce the CI
load per PR. They may still be optionally requested.
Reviewed-by: Alexander Moch <mail at alexmoch.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #19182
build: Use release version from META for git builds
This allows easier package upgrade management for package git builds.
The previous behavior had the release number be the number of patches
on top of a ZFS release or on top of master. This is an issue if
patches are removed, thus making the release version not monotonically
increasing. This won't happen if only using official commits because
the history is treated as immutable. But for those wanting to carry
patches that have not been accepted, this can happen.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Glenn Washburn <development at efficientek.com>
Closes #18953
Closes #19159
pkg: Fix rpm warnings about not being able to read its database
When building on many debian machines, the RPM database, by default at
/var/lib/rpm/rpmdb.sqlite, does not exist and can not be created when
running as an unprivileged user. This will cause any rpm command that
ties to access the database to print this obnoxious error message:
error: Unable to open sqlite database /var/lib/rpm/rpmdb.sqlite: unable to open database file
error: cannot open Packages index using sqlite - Operation not permitted (1)
error: cannot open Packages database in /var/lib/rpm
It appears as though this doesn't cause any failures and is really a
warning. However, it can be printed more than 20 times when making the
deb-kmod target.
Detect when the database file is not readable at configure time and
setup various rpm and rpmbuild invocations to use the root of the build
directory as the database directory path. The alien program internally
runs various rpm queries, however it provides no good way to tell those
[7 lines not shown]
deb: Print error message when build depends are not satified
An error message is already displayed by dpkg-buildpackage when build
dependencies are not met. However, the new error message shows the
exact command to use to satisfy those dependencies.
Rename the target native-deb-local to deb-local-common because its
rules are common to both deb and native-deb package builds. And make
nateive-deb-local depend on both deb-local-common and the new
native-deb-check-build-dep.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Glenn Washburn <development at efficientek.com>
Closes #19161
deb: Allow building when builddir is not srcdir
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Glenn Washburn <development at efficientek.com>
Closes #19160
rpm: Make dash compatible
On some systems, notaby Debian and many of its derivatives, /bin/sh is
a symlink to /usr/bin/dash. The scripts created for the various
rpmbuild stages are run with /bin/sh. The construct {a,b,c} does not
get expanded in dash, as it does in bash's sh.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Glenn Washburn <development at efficientek.com>
Closes #19158
FreeBSD: remove legacy ioctl support
When FreeBSD switched their ZFS to OpenZFS in 13, a compat layer was
included to allow older userspace to work with newer kernel modules and
vice-versa. Since FreeBSD 12 is now almost 3 years out of support, we
shouldn't need this compatibility support any longer, and can remove it.
The small versioned wrapping struct zfs_iocparm_t is still in use for
current ioctls (and honestly a pretty good idea), so that's retained,
and we're now explicitly enforcing the version and size fields are what
they're supposed to be.
Sponsored-by: TrueNAS
Reviewed-by: Kyle Evans <kevans at FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan.moeller at klarasystems.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19157
Propagate vdev state change on probe failure
Currently, when probe fails on some vdev, it would became
faulted, but the parent's vdev state won't be changed. So we
can have a situation, for example, when several disks in draid
vdev are faulted (domain failure), but draid vdev itself and
pool are online, which doesn't seem right.
Solution: set isopen argument to false when calling
vdev_set_state() from spa_async_fault_vdev(). It's not quite
clear why it was set to true in the first place when the code
was introduced at commit c4f2f1aa2d.
isopen argument to vdev_set_state() is normally set to true
when we are "in the process of opening children depth-first",
in which case we don't want to propagate the change of state
to the parent, but it doesn't seem to be the case here.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[2 lines not shown]
libzfs: do not iterate snapshots for a mountpoint changelist
The mountpoint property does not apply to snapshots, so change_one()
drops every snapshot it is given. Iterating over them while gathering
a mountpoint changelist therefore costs a full stats ioctl per snapshot
and never contributes an entry, which makes "zfs set mountpoint", "zfs
inherit mountpoint" and "zfs set volsize" scale with the number of
snapshots for no reason.
Iterate filesystems only. This also covers canmount, volsize and zoned,
since those are handled through the mountpoint changelist.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Ameer Hamza <ahamza at ixsystems.com>
Closes #19137
ZTS: Speed up replace_resilver_sit_out
Reduce replace_resilver_sit_out test time from 1:14 to 30sec.
- Use 4 "lanes" in zinject instead of 1.
- Use a 1M recordsize to speed up replacements.
- Use 1 second zfs_txg_timeout to help with replacement
Also, double the number or reads to more reliably hit the sit-out
condition. This test has been flaky in the past.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Moch <mail at alexmoch.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #19156
zstream: command line and man page cleanup
This PR makes several changes to the user-facing command line syntax and
documentation:
- The `decompress,` `drop_records,` and `recompress` subcommands now
allow naming a stream file on the command line, bringing them into
line with `dump` and `raw.` Reading from standard input remains an
option.
- Tests have been modified to exercise command-line specification of
input files.
- The man page has been updated and copy edited.
- Subcommands are presented in a consistent (alphabetical) order in the
man page, the usage message, and the source code.
- The man page and usage message now use identical names and
[71 lines not shown]
zed: fix failure domain detection for unavail case
When the domain fails and there is no traffic, zed might get
list.suspect event with the disks entering into UNAVAIL state,
instead of zfs.statechange events with the disks entering into
FAULTED state. We should handle that situation similarly and
and don't start resilvering, which would be only a huge waste
of CPU and i/o bandwidth resources.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
Related-to #18148
Closes #19154
rpm: Add configure caching if used in main configure run
When a user enables configure caching when running configure, eg. by
passing -C, enable configure caching in the corresponding rpm builds.
The configure cache from the initial configure run by the user can not
generally be used because the host and build values will be different.
Normally those values are left unset when invoking configure whereas
the configure from rpmbuild explicitly sets them.
However, the intent of the user to enable configure caching can still
be satisfied by using a cache file solely for rpm builds. This implies
that initially, when building an rpm package with configure caching
enabled, that the first two runs of configure will be uncached, the
user-initiated and rpmbuild initiated ones. The cache file is named
config.rpm.cache and is located in the root of the build directory.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Glenn Washburn <development at efficientek.com>
Closes #19111
Closes #19148
Make the spa_config_lock reader fast path lockless
Readers took scl_lock on every spa_config_enter()/exit(), which cost
~29% of CPU time in 4KB record write workloads. Fold the writer gate
into the reference count: SCL_COUNT_WRITER in the top bit of scl_count
lets a reader take a reference and test for a writer in one atomic
operation, and lets the last reader detect a waiting writer without a
second load. Writers still serialize on scl_lock, and priority readers
still take it, since they must tell a waiting writer from a holding one.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #19131
Avoid dp_lock in the dirty space accounting
dsl_pool_dirty_space() and dsl_pool_undirty_space() are called once per
logical block, from open context and from every ZIO completion, all
serializing on the single pool-wide dp_lock. On a 12-thread 32KB
overwrite benchmark they took 33% of CPU, almost all of it spinning.
The lock only protected the counters against each other, so make
dp_dirty_pertxg/dp_dirty_total and dp_sync_reserve_pertxg/_total plain
atomics. The subtractions clamp at zero via a CAS loop. Reading these
counters without the lock was already the norm.
The cv_signal() on every accounting event is only ever needed by
dmu_tx_wait(), which sleeps solely at the hard zfs_dirty_data_max wall,
so gate it on a count of waiters. Both sides need membar_sync(), since
the atomics used here provide no ordering. dmu_tx_wait() in turn only
registers as a waiter if it may really sleep, keeping dp_lock off the
soft throttle path taken by every delayed transaction.
[5 lines not shown]
freebsd: remove _KERNEL gates
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
linux: remove _KERNEL gates
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
icp: remove _KERNEL gates
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
lua: remove _KERNEL gates
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
nvpair: remove _KERNEL gates
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
zfs: remove _KERNEL gates
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
libspl: remove platform gates from platform headers
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
linux: remove platform gates from platform headers
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19153
