Displaying 1 50 of 4,913 commits (0.007s)

ZFS on Linux — contrib/dracut/90zfs zfs-load-key.sh.in mount-zfs.sh.in, etc/systemd/system zfs-import-cache.service.in zfs-import-scan.service.in

Run zfs load-key if needed in dracut

'zfs load-key -a' will only be called if needed.  If a dataset not
needed for boot does not have its key loaded (home directories for
example) boot can still continue.

zfs:AUTO was not working via dracut, so we still need the generator
script to do its thing.

Reviewed-by: Richard Yao <ryao at gentoo.org>
Reviewed-by: Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Matthew Thode <mthode at mthode.org>
Closes #6982
Closes #7004

ZFS on Linux — config deb.am

    Fix Debian packaging on ARMv7/ARM64
    
    When building packages on Debian-based systems specify the target
    architecture used by 'alien' to convert .rpm packages into .deb: this
    avoids detecting an incorrect value which results in the following
    errors:
    
    <package>.aarch64.rpm is for architecture aarch64 ; the package cannot be built on 
this system
    <package>.armv7l.rpm is for architecture armel ; the package cannot be built on this 
system
    
    Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
    Reviewed-by: George Melikov <mail at gmelikov.ru>
    Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
    Closes #7046
    Closes #7058
Delta File
+6 -3 config/deb.am
+6 -3 1 file

ZFS on Linux — module/zfs mmp.c

Emit an error message before MMP suspends pool

In mmp_thread(), emit an MMP specific error message before calling
zio_suspend() so that the administrator will understand why the pool
is being suspended.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: John L. Hammond <john.hammond at intel.com>
Closes #7048
Delta File
+5 -0 module/zfs/mmp.c
+5 -0 1 file

ZFS on Linux — cmd/zed/zed.d zed-functions.sh, scripts commitcheck.sh zloop.sh

Fix shellcheck v0.4.6 warnings

Resolve new warnings reported after upgrading to shellcheck
version 0.4.6.  This patch contains no functional changes.

* egrep is non-standard and deprecated. Use grep -E instead. [SC2196]
* Check exit code directly with e.g. 'if mycmd;', not indirectly
  with $?.  [SC2181]  Suppressed.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7040

ZFS on Linux — man/man5 zfs-module-parameters.5

Remove l2arc_nocompress from zfs-module-parameters(5)

Parameter was removed in d3c2ae1c0806
(OpenZFS 6950 - ARC should cache compressed data)

Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: DHE <git at dehacked.net>
Closes #7043
Delta File
+0 -11 man/man5/zfs-module-parameters.5
+0 -11 1 file

ZFS on Linux — . copy-builtin

Fix copy-builtin to work with ASAN patch

Commit fed90353 didn't fully update the copy-builtin script
as needed to perform in-kernel builds.  Add the missing
options and flags.

Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Thode <mthode at mthode.org>
Closes #7033
Closes #7037
Delta File
+4 -2 copy-builtin
+4 -2 1 file

ZFS on Linux — cmd/ztest ztest.c, include/sys zfs_context.h

Force ztest to always use /dev/urandom

For ztest, which is solely for testing, using a pseudo random
is entirely reasonable.  Using /dev/urandom ensures the system
entropy pool doesn't get depleted thus stalling the testing.
This is a particular problem when testing in VMs.

Reviewed-by: Tim Chase <tim at chase2k.com>
Reviewed by: Thomas Caputi <tcaputi at datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7017
Closes #7036

ZFS on Linux — cmd/raidz_test Makefile.am, cmd/zdb Makefile.am

Support -fsanitize=address with --enable-asan

When --enable-asan is provided to configure then build all user
space components with fsanitize=address.  For kernel support
use the Linux KASAN feature instead.

https://github.com/google/sanitizers/wiki/AddressSanitizer

When using gcc version 4.8 any test case which intentionally
generates a core dump will fail when using --enable-asan.
The default behavior is to disable core dumps and only newer
versions allow this behavior to be controled at run time with
the ASAN_OPTIONS environment variable.

Additionally, this patch includes some build system cleanup.

* Rules.am updated to set the minimum AM_CFLAGS, AM_CPPFLAGS,
  and AM_LDFLAGS.  Any additional flags should be added on a
  per-Makefile basic.  The --enable-debug and --enable-asan
  options apply to all user space binaries and libraries.

* Compiler checks consolidated in always-compiler-options.m4
  and renamed for consistency.


    [27 lines not shown]

ZFS on Linux — tests/zfs-tests/tests/functional/history history_004_pos.ksh

Disable history_004_pos

Occasionally observed failure of history_004_pos due to the test
case not being 100% reliable.  In order to prevent false positives
disable this test case until it can be made reliable.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Issue #7026
Closes #7028

ZFS on Linux — config kernel-vfs-rw-iterate.m4

Fix incompatibility with Reiser4 patched kernels

In ZFSOnLinux, our sources and build system are self contained such that
we do not need to make changes to the Linux kernel sources. Reiser4 on
the other hand exists solely as a kernel tree patch and opts to make
changes to the kernel rather than adapt to it. After Linux 4.1 made a
VFS change that replaced new_sync_read with do_sync_read, Reiser4's
maintainer decided to modify the kernel VFS to export the old function.
This caused our autotools check to misidentify the kernel API as
predating Linux 4.1 on kernels that have been patched with Reiser4
support, which breaks our build.

Reiser4 really should be patched to stop doing this, but lets modify our
check to be more strict to help the affected users of both filesystems.

Also, we were not checking the types of arguments and return value of
new_sync_read() and new_sync_write() . Lets fix that too.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Richard Yao <ryao at gentoo.org>
Closes #6241
Closes #7021
Delta File
+11 -3 config/kernel-vfs-rw-iterate.m4
+11 -3 1 file

ZFS on Linux — include/sys vdev_raidz_impl.h

Revert raidz_map and _col structure types

As part of the refactoring of ab9f4b0b824ab4cc64a4fa382c037f4154de12d6,
several uint64_t-s and uint8_t-s were changed to other types.  This
caused ZoL github issue #6981, an overflow of a size_t on a 32-bit ARM
machine.  In absense of any strong motivation for the type changes, this
simply puts them back, modulo the changes accumulated for ABD.

Compile-tested on amd64 and run-tested on armhf.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Gvozden Neskovic <neskovic at gmail.com>
Signed-off-by: Nathaniel Wesley Filardo <nwf at cs.jhu.edu>
Closes #6981
Closes #7023
Delta File
+17 -17 include/sys/vdev_raidz_impl.h
+17 -17 1 file

ZFS on Linux — man/man5 zfs-module-parameters.5

Fix percentage styling in zfs-module-parameters.5

Replace "percent" with "%", add bold to default values.

Reviewed-by: bunder2015 <omfgbunder at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Amanakis <gamanakis at gmail.com>
Closes #7018
Delta File
+17 -17 man/man5/zfs-module-parameters.5
+17 -17 1 file

ZFS on Linux — .github codecov.yml

Reduce codecov PR comments

Attempt to reduce the number of comments posted by codecov
to PR requests.  Based on the codecov documenation setting
"require_changes=yes" and "behavior=once" should result in
a single comment under most circumstances.

https://docs.codecov.io/v4.3.6/docs/pull-request-comments

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Issue #7022
Closes #7025
Delta File
+2 -1 .github/codecov.yml
+2 -1 1 file

ZFS on Linux — cmd/zhack zhack.c

zhack: fix getopt return type

This fixes zhack's command processing on ARM.  On ARM char
is unsigned, and so, in promotion to an int, it will never
compare equal to -1.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Nathaniel Wesley Filardo <nwf at cs.jhu.edu>
Closes #7016
Delta File
+3 -3 cmd/zhack/zhack.c
+3 -3 1 file

ZFS on Linux — module/zfs zfs_dir.c

Use zap_count instead of cached z_size for unlink

As a performance optimization Lustre does not strictly update
the SA_ZPL_SIZE when adding/removing from non-directory entries.
This results in entries which cannot be removed through the ZPL
layer even though the ZAP is empty and safe to remove.

Resolve this issue by checking the zap_count() directly instead
on relying on the cached SA_ZPL_SIZE.  Micro-benchmarks show no
significant performance impact due to the additional overhead
of using zap_count().

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev at intel.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7019
Delta File
+15 -1 module/zfs/zfs_dir.c
+15 -1 1 file

ZFS on Linux — include/sys arc.h, module/zfs arc.c dbuf.c

Fix ARC hit rate

When the compressed ARC feature was added in commit d3c2ae1
the method of reference counting in the ARC was modified.  As
part of this accounting change the arc_buf_add_ref() function
was removed entirely.

This would have be fine but the arc_buf_add_ref() function
served a second undocumented purpose of updating the ARC access
information when taking a hold on a dbuf.  Without this logic
in place a cached dbuf would not migrate its associated
arc_buf_hdr_t to the MFU list.  This would negatively impact
the ARC hit rate, particularly on systems with a small ARC.

This change reinstates the missing call to arc_access() from
dbuf_hold() by implementing a new arc_buf_access() function.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Tim Chase <tim at chase2k.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6171

    [2 lines not shown]
Delta File
+51 -1 module/zfs/arc.c
+3 -1 module/zfs/dbuf.c
+1 -0 include/sys/arc.h
+55 -2 3 files

ZFS on Linux — cmd/zpool zpool_vdev.c, tests/runfiles linux.run

Fix 'zpool add' handling of nested interior VDEVs

When replacing a faulted device which was previously handled by a spare
multiple levels of nested interior VDEVs will be present in the pool
configuration; the following example illustrates one of the possible
situations:

   NAME                          STATE     READ WRITE CKSUM
   testpool                      DEGRADED     0     0     0
     raidz1-0                    DEGRADED     0     0     0
       spare-0                   DEGRADED     0     0     0
         replacing-0             DEGRADED     0     0     0
           /var/tmp/fault-dev    UNAVAIL      0     0     0  cannot open
           /var/tmp/replace-dev  ONLINE       0     0     0
         /var/tmp/spare-dev1     ONLINE       0     0     0
       /var/tmp/safe-dev         ONLINE       0     0     0
   spares
     /var/tmp/spare-dev1         INUSE     currently in use

This is safe and allowed, but get_replication() needs to handle this
situation gracefully to let zpool add new devices to the pool.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>

    [2 lines not shown]

ZFS on Linux — include/sys dmu.h dmu_tx.h, module/zfs dmu_tx.c

Call commit callbacks from the tail of the list

Our zfs backed Lustre MDT had soft lockups while under heavy metadata
workloads while handling transaction callbacks from osd_zfs.

The problem is zfs is not taking advantage of the fast path in
Lustre's trans callback handling, where Lustre will skip the calls
to ptlrpc_commit_replies() when it already saw a higher transaction
number.

This patch corrects this, it also has a positive impact on metadata
performance on Lustre with osd_zfs, plus some cleanup in the headers.

A similar issue for ext4/ldiskfs is described on:
https://jira.hpdd.intel.com/browse/LU-6527

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Li Dongyang <dongyang.li at anu.edu.au>
Closes #6986

ZFS on Linux — . io_spa- spa_stats.io_history

Remove empty files accidentally added by a8b2e306

This patch simply removes 2 empty files that were accidentally
added a part of the scrub priority patch.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #6990
Delta File
+0 -0 io_spa-
+0 -0 spa_stats.io_history
+0 -0 2 files

ZFS on Linux — . spa_stats.io_history io_spa-, include/sys trace_arc.h zio.h

Support re-prioritizing asynchronous prefetches

When sequential scrubs were merged, all calls to arc_read()
(including prefetch IOs) were given ZIO_PRIORITY_ASYNC_READ.
Unfortunately, this behaves badly with an existing issue where
prefetch IOs cannot be re-prioritized after the issue. The
result is that synchronous reads end up in the same vdev_queue
as the scrub IOs and can have (in some workloads) multiple
seconds of latency.

This patch incorporates 2 changes. The first ensures that all
scrub IOs are given ZIO_PRIORITY_SCRUB to allow the vdev_queue
code to differentiate between these I/Os and user prefetches.
Second, this patch introduces zio_change_priority() to provide
the missing capability to upgrade a zio's priority.

Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #6921
Closes #6926

ZFS on Linux — cmd/vdev_id vdev_id, man/man5 vdev_id.conf.5

vdev_id: new slot type ses

This extends vdev_id to support a new slot type, ses, for SCSI Enclosure
Services.  With slot type ses, the disk slot numbers are determined by
using the device slot number reported by sg_ses for the device with
matching SAS address, found by querying all available enclosures.

This is primarily of use on systems with a deficient driver omitting
support for bay_identifier in /sys/devices.  In my testing, I found that
the existing slot types of port and id were not stable across disk
replacement, so an alternative was required.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Simon Guest <simon.guest at tesujimath.org>
Closes #6956
Delta File
+17 -0 cmd/vdev_id/vdev_id
+7 -1 man/man5/vdev_id.conf.5
+24 -1 2 files

ZFS on Linux — cmd/arc_summary arc_summary.py, tests/zfs-tests/tests/functional/cli_user/misc arc_summary_001_pos.ksh

Handle broken pipes in arc_summary

Using a command similar to 'arc_summary.py | head' causes
a broken pipe exception. Gracefully exit in the case of a
broken pipe in arc_summary.py.

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #6965
Closes #6969

ZFS on Linux — cmd/arc_summary arc_summary.py, tests/runfiles linux.run

Handle invalid options in arc_summary

If an invalid option is provided to arc_summary.py we handle any error
thrown from the getopt Python module and print the usage help message.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #6983

ZFS on Linux — cmd/zdb zdb.c, tests/zfs-tests/tests/functional/cli_root/zpool_create create-o_ashift.ksh

ZTS: Fix create-o_ashift test case

The function that fills the uberblock ring buffer on every device label
has been reworked to avoid occasional failures caused by a race
condition that prevents 'zpool sync' from writing some uberblock
sequentially: this happens when the pool sync ioctl dispatch code calls
txg_wait_synced() while we're already waiting for a TXG to sync.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #6924
Closes #6977

ZFS on Linux — . META, rpm/generic zfs-kmod.spec.in zfs.spec.in

Tag zfs-0.7.5

META file and changelog updated.

Signed-off-by: Tony Hutter <hutter2 at llnl.gov>

ZFS on Linux — module/zfs spa.c, tests/zfs-tests/tests/functional/mmp mmp.kshlib mmp_active_import.ksh

Fix multihost stale cache file import

When the multihost property is enabled it should be impossible to
import an active pool even using the force (-f) option.  This patch
prevents a forced import from succeeding when importing with a
stale cache file.

The root cause of the problem is that the kernel modules trusted
the hostid provided in configuration.  This is always correct when
the configuration is generated by scanning for the pool.  However,
when using an existing cache file the hostid could be stale which
would result in the activity check being skipped.

Resolve the issue by always using the hostid read from the label
configuration where the best uberblock was found.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6933
Closes #6971

ZFS on Linux — module/zfs spa.c, tests/zfs-tests/tests/functional/mmp mmp.kshlib mmp_active_import.ksh

Fix multihost stale cache file import

When the multihost property is enabled it should be impossible to
import an active pool even using the force (-f) option.  This patch
prevents a forced import from succeeding when importing with a
stale cache file.

The root cause of the problem is that the kernel modules trusted
the hostid provided in configuration.  This is always correct when
the configuration is generated by scanning for the pool.  However,
when using an existing cache file the hostid could be stale which
would result in the activity check being skipped.

Resolve the issue by always using the hostid read from the label
configuration where the best uberblock was found.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6933
Closes #6971

ZFS on Linux — etc/systemd/system 50-zfs.preset.in

Enable zfs-import.target in systemd preset (#6968)

Cherry picked line from PR #6822, this enables the new
target introduced in PR #6764.

Signed-off-by: Antonio Russo <antonio.e.russo at gmail.com>

ZFS on Linux — cmd/mount_zfs Makefile.am, contrib/dracut/90zfs module-setup.sh.in Makefile.am

Honor --with-mounthelperdir where applicable

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #6962

ZFS on Linux — config user-systemd.m4 zfs-build.m4, rpm/generic zfs.spec.in

Fix --with-systemd on Debian-based distributions (#6963)

These changes propagate the "--with-systemd" configure option to the
RPM spec file, allowing Debian-based distributions to package
systemd-related files.

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #6591
Closes #6963

ZFS on Linux — lib/libspl/include/sys frame.h Makefile.am

Remove lib/libspl/include/sys/frame.h

The functionality provided by this header is not required by any
of the ZFS user space code.  Minimal functionality was provided
in commit c28a677 which added include/sys/frame.h.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6960
Closes #6972

ZFS on Linux — rpm/generic zfs.spec.in

Add zfs-import.target services in spec file

Add missing zfs-import.target to list of systemd services in zfs
RPM spec file.

Reviewed-by: Niklas Wagner <Skaro at Skaronator.com>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Ralf Ertzinger <ralf at skytale.net>
Issue #6953
Closes #6955
Delta File
+1 -1 rpm/generic/zfs.spec.in
+1 -1 1 file

ZFS on Linux — lib/libzfs libzfs_pool.c

OpenZFS 8897 - zpool online -e fails assertion when run on non-leaf vdevs

Authored by: Yuri Pankov <yuri.pankov at nexenta.com>
Reviewed by: Toomas Soome <tsoome at me.com>
Reviewed by: Igor Kozhukhov <igor at dilos.org>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/8897
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/9a551dd645
Closes #7030
Delta File
+4 -2 lib/libzfs/libzfs_pool.c
+4 -2 1 file

ZFS on Linux — cmd/zed/agents zfs_mod.c zfs_retire.c, tests/runfiles linux.run

Various ZED fixes

* Teach ZED to handle spares usingi the configured ashift: if the zpool
   'ashift' property is set then ZED should use its value when kicking
   in a hotspare; with this change 512e disks can be used as spares
   for VDEVs that were created with ashift=9, even if ZFS natively
   detects them as 4K block devices.

 * Introduce an additional auto_spare test case which verifies that in
   the face of multiple device failures an appropiate number of spares
   are kicked in.

 * Fix zed_stop() in "libtest.shlib" which did not correctly wait the
   target pid.

 * Fix ZED crashing on startup caused by a race condition in libzfs
   when used in multi-threaded context.

 * Convert ZED over to using the tpool library which is already present
   in the Illumos FMA code.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #2562
Closes #6858

ZFS on Linux — tests/zfs-tests/tests/functional/vdev_zaps vdev_zaps_004_pos.ksh

Disable vdev_zaps_004_pos

Occasionally observed failure of vdev_zaps_004_pos due to the test
case not being 100% reliable.  In order to prevent false positives
disable this test case until it can be made reliable.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Issue #6935
Closes #6936

ZFS on Linux — cmd/zed/agents zfs_diagnosis.c, cmd/zpool zpool_main.c

Fix unused variable warnings

Resolved unused variable warnings observed after restricting
-Wno-unused-but-set-variable to only libzfs and libzpool.

Reviewed-by: DHE <git at dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6941

ZFS on Linux — include/sys zil_impl.h zio.h, man/man5 zfs-module-parameters.5

    OpenZFS 8909 - 8585 can cause a use-after-free kernel panic
    
    Authored by: Prakash Surya <prakash.surya at delphix.com>
    Reviewed by: John Kennedy <jwk404 at gmail.com>
    Reviewed by: Matthew Ahrens <mahrens at delphix.com>
    Reviewed by: George Wilson <george.wilson at delphix.com>
    Reviewed by: Brad Lewis <brad.lewis at delphix.com>
    Reviewed by: Igor Kozhukhov <igor at dilos.org>
    Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
    Approved by: Robert Mustacchi <rm at joyent.com>
    Ported-by: Prakash Surya <prakash.surya at delphix.com>
    
    PROBLEM
    =======
    
    There's a race condition that exists if `zil_free_lwb` races with either
    `zil_commit_waiter_timeout` and/or `zil_lwb_flush_vdevs_done`.
    
    Here's an example panic due to this bug:
    
        > ::status
        debugging crash dump vmcore.0 (64-bit) from ip-10-110-205-40
        operating system: 5.11 dlpx-5.2.2.0_2017-12-04-17-28-32b6ba51fb (i86pc)
        image uuid: 4af0edfb-e58e-6ed8-cafc-d3e9167c7513
        panic message:

    [93 lines not shown]

ZFS on Linux — config kernel-objtool.m4, include/sys frame.h Makefile.am

Suppress incorrect objtool warnings

Suppress incorrect warnings from versions of objtool which are not
aware of x86 EVEX prefix instructions used for AVX512.

  module/zfs/vdev_raidz_math_avx512bw.o: warning:
  objtool: <func+offset>: can't find jump dest instruction at .text

Reviewed-by: Don Brady <don.brady at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6928

ZFS on Linux — scripts dkms.mkconf

Enable QAT support in zfs-dkms RPM

Enable QAT accelerated gzip compression in zfs-dkms RPM package when
environment variant ICP_ROOT is set to QAT drive source code folder
and QAT hardware presence.  Otherwise, use default gzip compression.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: David Qian <david.qian at intel.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6932
Delta File
+6 -0 scripts/dkms.mkconf
+6 -0 1 file

ZFS on Linux — cmd/ztest ztest.c

Fix ztest_verify_dnode_bt() test case

In ztest_verify_dnode_bt the ztest_object_lock must be held in
order to safely verify the unused bonus space.

Reviewed-by: DHE <git at dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6941
Delta File
+5 -1 cmd/ztest/ztest.c
+5 -1 1 file

ZFS on Linux — cmd/zpool zpool_main.c

Fix segfault in zpool iostat when adding VDEVs

Fix a segfault when running 'zpool iostat -v 1' while adding
a VDEV.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #6748
Closes #6872
Delta File
+15 -7 cmd/zpool/zpool_main.c
+15 -7 1 file

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zpool_create create-o_ashift.ksh

Disable create-o_ashift

Occasionally observed failure of create-o_ashift due to the test
case not being 100% reliable.  In order to prevent false positives
disable this test case until it can be made reliable.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Issue #6924
Closes #6925

ZFS on Linux — man/man8 zpool.8

OpenZFS 8899 - zpool list property documentation doesn't match actual behaviour

Authored by: Yuri Pankov <yuri.pankov at nexenta.com>
Reviewed by: Alexander Pyhalov <alp at rsu.ru>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/8899
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/b0e142e57d
Closes #7032
Delta File
+5 -9 man/man8/zpool.8
+5 -9 1 file

ZFS on Linux — lib/libzfs libzfs_dataset.c, man/man8 zfs.8

OpenZFS 8898 - creating fs with checksum=skein on the boot pools fails ungracefully

Authored by: Yuri Pankov <yuri.pankov at nexenta.com>
Reviewed by: Toomas Soome <tsoome at me.com>
Reviewed by: Andy Stormont <astormont at racktopsystems.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/8898
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/9fa2266d9a
Closes #7031
Delta File
+5 -1 lib/libzfs/libzfs_dataset.c
+5 -1 man/man8/zfs.8
+10 -2 2 files

ZFS on Linux — cmd/ztest ztest.c, include/sys trace_zil.h zil_impl.h

OpenZFS 8585 - improve batching done in zil_commit()

Authored by: Prakash Surya <prakash.surya at delphix.com>
Reviewed by: Brad Lewis <brad.lewis at delphix.com>
Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Prakash Surya <prakash.surya at delphix.com>

Problem
=======

The current implementation of zil_commit() can introduce significant
latency, beyond what is inherent due to the latency of the underlying
storage. The additional latency comes from two main problems:

 1. When there's outstanding ZIL blocks being written (i.e. there's
    already a "writer thread" in progress), then any new calls to
    zil_commit() will block waiting for the currently oustanding ZIL
    blocks to complete. The blocks written for each "writer thread" is
    coined a "batch", and there can only ever be a single "batch" being
    written at a time. When a batch is being written, any new ZIL
    transactions will have to wait for the next batch to be written,
    which won't occur until the current batch finishes.

    [102 lines not shown]
Delta File
+1,280 -262 module/zfs/zil.c
+163 -71 include/sys/trace_zil.h
+62 -11 include/sys/zil_impl.h
+25 -8 module/zfs/zio.c
+11 -3 lib/libspl/include/sys/time.h
+7 -5 module/zfs/zfs_vnops.c
+39 -18 9 files not shown
+1,587 -378 15 files

ZFS on Linux — module/zfs zfs_dir.c

Fix NFS sticky bit permission denied error

When zfs_sticky_remove_access() was originally adapted for Linux
a typo was made which altered the intended behavior.  As described
in the block comment, the intended behavior is that permission
should be granted when the entry is a regular file and you have
write access.  That is, S_ISREG should have been used instead of
S_ISDIR.

Restricting permission to regular files made good sense for older
systems where setting the bit on executable files would instruct
the system to save the program's text segment on the swap device.

On modern systems this behavior has been replaced by the sticky
bit acting as a restricted deletion flag and the plain file
restriction has been relaxed.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6889
Closes #6910
Delta File
+2 -3 module/zfs/zfs_dir.c
+2 -3 1 file

ZFS on Linux — contrib/initramfs/hooks zfs.in

Add /usr/bin/env to COPY_EXEC_LIST initramfs hook

5dc1ff29 changed the user space program to mount a zfs snapshot
from /bin/sh to /usr/bin/env.  If the executable is not present
in the initramfs then snapshots cannot be automounted.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: James Dingwall <james.dingwall at zynstra.com>
Closes #5360
Closes #6913
Delta File
+1 -0 contrib/initramfs/hooks/zfs.in
+1 -0 1 file

ZFS on Linux — cmd/zpool zpool_vdev.c, tests/zfs-tests/tests/functional/cli_root/zpool_add zpool_add_010_pos.ksh

Fix 'zpool create|add' replication level check

When the pool configuration contains a hole due to a previous device
removal ignore this top level vdev.  Failure to do so will result in
the current configuration being assessed to have a non-uniform
replication level and the expected warning will be disabled.

The zpool_add_010_pos test case was extended to cover this scenario.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6907
Closes #6911

ZFS on Linux — include/sys zil.h, module/zfs zil.c

Preserve itx alloc size for zio_data_buf_free()

Using zio_data_buf_alloc() to allocate the itx's may be unsafe
because the itx->itx_lr.lrc_reclen field is not constant from
allocation to free.  Using a different itx->itx_lr.lrc_reclen
size in zio_data_buf_free() can result in the allocation being
returned to the wrong kmem cache.

This issue can be avoided entirely by storing the allocation size
in itx->itx_size and using that for zio_data_buf_free().

Reviewed by: Prakash Surya <prakash.surya at delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #6912
Delta File
+5 -2 module/zfs/zil.c
+1 -0 include/sys/zil.h
+6 -2 2 files

ZFS on Linux — include/sys/fs zfs.h, module/zfs spa_misc.c

Unbreak the scan status ABI

When d4a72f23 was merged, pss_pass_issued was incorrectly
added to the middle of the pool_scan_stat_t structure
instead of the end. This patch simply corrects this issue.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #6909
Delta File
+5 -5 module/zfs/spa_misc.c
+1 -1 include/sys/fs/zfs.h
+6 -6 2 files