Displaying 1 50 of 5,324 commits (0.007s)

ZFS on Linux — module/zfs vdev_disk.c

vdev_disk_error() prints ASCII SOH to debug log

Currently vdev_disk_error() prepends its messages sent to the internal
ZFS debug log with KERN_WARNING, which is currently defined as follows:

   #define KERN_SOH      "\001"
   #define KERN_WARNING  KERN_SOH "4"

Since "\001" (ASCII Start Of Header) is not printable this results in
weird characters displayed when inspecting the debug log. This commit
simply removes this superfluous prefix passed to zfs_dbgmsg().

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7936
Delta File
+3 -4 module/zfs/vdev_disk.c
+3 -4 1 file

ZFS on Linux — man/man8 zpool.8

Fix reference to zpool-features(5)

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Signed-off-by: DHE <git at dehacked.net>
Closes #7938
Delta File
+1 -1 man/man8/zpool.8
+1 -1 1 file

ZFS on Linux — module/zfs spa_misc.c

Add limits to spa_slop_shift tunable

This change adds limits to the possible spa_slop_shift values set via
the sysfs interface. Accepted values are from a minimum of 1 to a
maximum of 31 (inclusive): these limits are based on the following
values observed on a 128PB file-vdev test pool:

spa_slop_shift=1, spa_get_slop_space=63.5PiB
spa_slop_shift=2, spa_get_slop_space=31.8PiB
spa_slop_shift=3, spa_get_slop_space=15.9PiB
spa_slop_shift=4, spa_get_slop_space=7.9PiB
spa_slop_shift=5, spa_get_slop_space=4PiB
spa_slop_shift=6, spa_get_slop_space=2PiB
...
spa_slop_shift=25, spa_get_slop_space=4GiB
spa_slop_shift=26, spa_get_slop_space=2GiB
spa_slop_shift=27, spa_get_slop_space=1016MiB
spa_slop_shift=28, spa_get_slop_space=508MiB
spa_slop_shift=29, spa_get_slop_space=254MiB
spa_slop_shift=30, spa_get_slop_space=128MiB
spa_slop_shift=31, spa_get_slop_space=128MiB
spa_slop_shift=32, spa_get_slop_space=128MiB

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>

    [3 lines not shown]
Delta File
+22 -1 module/zfs/spa_misc.c
+22 -1 1 file

ZFS on Linux — . NEWS Makefile.am

Add NEWS file

I received a request for a NEWS file. That needs to be handled by Tony
and Brian, but for now, we can at least provide one that provides a link
to github so that users who expect NEWS files from their packages will
know where to go for release information.

Reviewed-by: Neal Gompa <ngompa at datto.com>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Richard Yao <ryao at gentoo.org>
Closes #7918
Delta File
+3 -0 NEWS
+1 -1 Makefile.am
+4 -1 2 files

ZFS on Linux — cmd/zstreamdump zstreamdump.c

zstreamdump dumps core printing truncated nvlist

This change prevents zstreamdump from crashing when trying to print
invalid nvlist data (DRR_BEGIN record) from a truncated send stream.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7917
Delta File
+5 -3 cmd/zstreamdump/zstreamdump.c
+5 -3 1 file

ZFS on Linux — man/man5 zpool-features.5

Fix allocation_classes GUID in zpool-features(5)

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Signed-off-by: DHE <git at dehacked.net>
Closes #7920
Delta File
+1 -1 man/man5/zpool-features.5
+1 -1 1 file

ZFS on Linux — .github CONTRIBUTING.md

Add new wiki page to CONTRIBUTING

A new wiki page has been added for users who may be new to Git or
GitHub.  Adding link to CONTRIBUTING.

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: Gregor Kopka <gregor at kopka.net>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7923
Closes #7915
Delta File
+1 -0 .github/CONTRIBUTING.md
+1 -0 1 file

ZFS on Linux — man/man8 zfs.8 zpool.8

Man page fixes - zpool/zfs optional parameters

The man pages for zpool and zfs (get command)
listed the pool/dataset parameter as required,
but these are optional. Fixed that.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Gregor Kopka <gregor at kopka.net>
Closes #7916
Delta File
+2 -2 man/man8/zfs.8
+2 -2 man/man8/zpool.8
+4 -4 2 files

ZFS on Linux — man/man8 zpool.8

Clarify 'zpool remove' restrictions

Update zpool(8) to clarify what type of vdevs may be safely
removed and that the existence of any top-level raidz device
which is part of the primary pool will prevent device removal.

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Signed-off-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7880
Closes #7893
Delta File
+8 -4 man/man8/zpool.8
+8 -4 1 file

ZFS on Linux — cmd/zpool zpool_main.c, tests/zfs-tests/tests/functional/cli_root/zpool_create zpool_create_003_pos.ksh

zpool should detect invalid fs property on create

This change improve the handling of invalid filesystem properties when
specified at pool creation: this is useful when 'zpool create -n'
(dry run) is executed to detect invalid fs-level options (-O) before
the actual command is run.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7620
Closes #7878

ZFS on Linux — tests/test-runner/bin zts-report.py, tests/zfs-tests/tests/functional/removal removal_resume_export.ksh

Add removal_resume_export to zts-report.py

Add the removal_resume_export test case to the possible failure
section of the zts-report.py and reference the Github issue.  In
the CI environment this test has proven to be unreliable due to
the way it detects the removal thread.  This is a flaw in the test
and not device removal so update the result summary accordingly.

Additionally, increase the allowed timeout in an effort to reduce
the observed rate of false positves.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7895
Issue #7894

ZFS on Linux — module/zfs spa.c, tests/runfiles linux.run

zpool split can create a corrupted pool

Added vdev_resilver_needed() check to verify VDEVs are fully
synced, so that after split the new pool will not be corrupted.

Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Roman Strashkin <roman.strashkin at nexenta.com>
Closes #7865
Closes #7881

ZFS on Linux — . META

Tag 0.8.0-rc1

Major new features:
- Native encryption
- Device removal
- Allocation classes
- Pool checkpoints
- Sequential scrub and resilver
- Project quota
- Channel programs
- Direct IO

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Delta File
+3 -3 META
+3 -3 1 file

ZFS on Linux — include/sys zfs_sysfs.h, module/zcommon zfeature_common.c zprop_common.c

Fix in-kernel sysfs entries

The recent sysfs zfs properties feature breaks the in-kernel
builds of zfs (sans module).  When not built as a module add
the sysfs entries under /sys/fs/zfs/.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Don Brady <don.brady at delphix.com>
Closes #7868
Closes #7872

ZFS on Linux — module/zfs zfs_sysfs.c

Fix zfs_sysfs_live test failure

The ZTS zfs_sysfs_live test fails occasionally due to an uninitialized
string on an error path.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Don Brady <don.brady at delphix.com>
Closes #7869
Delta File
+4 -2 module/zfs/zfs_sysfs.c
+4 -2 1 file

ZFS on Linux — cmd/zfs zfs_main.c, man/man8 zfs.8

Fix 'zfs allow' for create time permissions

When no permission set is defined for a dataset the create time
permissions are incorrectly shown as if they were a permission set.
This change simply correct how allow permissions are displayed.

This commit also fixes a small manpage formatting issue and adds the
"zfs_allow_003_pos" test case to the ZFS Test Suite.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7519
Closes #7860

ZFS on Linux — . configure.ac, cmd/zdb zdb.c

Pool allocation classes

Allocation Classes add the ability to have allocation classes in a
pool that are dedicated to serving specific block categories, such
as DDT data, metadata, and small file blocks. A pool can opt-in to
this feature by adding a 'special' or 'dedup' top-level VDEV.

Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: Alek Pinchuk <apinchuk at datto.com>
Reviewed-by: Håkan Johansson <f96hajo at chalmers.se>
Reviewed-by: Andreas Dilger <andreas.dilger at chamcloud.com>
Reviewed-by: DHE <git at dehacked.net>
Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: Gregor Kopka <gregor at kopka.net>
Reviewed-by: Kash Pande <kash at tripleback.net>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens at delphix.com>
Signed-off-by: Don Brady <don.brady at delphix.com>
Closes #5182
Delta File
+204 -81 cmd/zpool/zpool_main.c
+183 -28 module/zfs/vdev.c
+170 -17 cmd/ztest/ztest.c
+84 -66 module/zfs/metaslab.c
+105 -23 cmd/zdb/zdb.c
+103 -10 cmd/zpool/zpool_vdev.c
+1,476 -56 51 files not shown
+2,325 -281 57 files

ZFS on Linux — module/zfs zfs_ctldir.c

Correctly handle errors from kern_path

As a regular kernel function, kern_path() returns errors as negative
errnos, such as -ELOOP. zfsctl_snapdir_vget() must convert these into
the positive errnos used throughout the ZFS code when it returns them
to other ZFS functions so that the ZFS code properly sees them as
errors.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Chris Siebenmann <cks.git01 at cs.toronto.edu>
Closes #7764
Closes #7864
Delta File
+1 -1 module/zfs/zfs_ctldir.c
+1 -1 1 file

ZFS on Linux — module/zfs arc.c

Added recalculation of ARC stats mid-eviction

Re-adds a recalculation step for the ARC stats after the MRU
eviction so that we don't pathologically attempt to evict the MFU.

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Authored-by: Mark Johnston <markj at freebsd.org>
Signed-off-by: Rich Ercolani <rincebrain at gmail.com>
Closes #7855
Delta File
+7 -0 module/zfs/arc.c
+7 -0 1 file

ZFS on Linux — module/zfs zfs_ctldir.c

Revert "Update zfs_admin_snapshot default value (disabled)"

This reverts commit a6214a0ae9e78d3cac0e495e2fcf7af0858a872f.
Disabling zfs_admin_snapshot by default results in multiple ZTS
tests failing which depend on this functionality.  Revert this
change until the relevant test cases can be updated.

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Issue #7838
Delta File
+1 -1 module/zfs/zfs_ctldir.c
+1 -1 1 file

ZFS on Linux — module/zfs zfs_ctldir.c

Update zfs_admin_snapshot default value (disabled)

It's disabled by default, update code to reflect
the documentation.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Gregor Kopka <gregor at kopka.net>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #7835
Closes #7838
Delta File
+1 -1 module/zfs/zfs_ctldir.c
+1 -1 1 file

ZFS on Linux — . configure.ac, cmd/zfs zfs_main.c

Add basic zfs ioc input nvpair validation

We want newer versions of libzfs_core to run against an existing
zfs kernel module (i.e. a deferred reboot or module reload after
an update).

Programmatically document, via a zfs_ioc_key_t, the valid arguments
for the ioc commands that rely on nvpair input arguments (i.e. non
legacy commands from libzfs_core). Automatically verify the expected
pairs before dispatching a command.

This initial phase focuses on the non-legacy ioctls. A follow-on
change can address the legacy ioctl input from the zfs_cmd_t.

The zfs_ioc_key_t for zfs_keys_channel_program looks like:

static const zfs_ioc_key_t zfs_keys_channel_program[] = {
       {"program",     DATA_TYPE_STRING,               0},
       {"arg",         DATA_TYPE_UNKNOWN,              0},
       {"sync",        DATA_TYPE_BOOLEAN_VALUE,        ZK_OPTIONAL},
       {"instrlimit",  DATA_TYPE_UINT64,               ZK_OPTIONAL},
       {"memlimit",    DATA_TYPE_UINT64,               ZK_OPTIONAL},
};

Introduce four input errors to identify specific input failures

    [12 lines not shown]

ZFS on Linux — . configure.ac, include zfeature_common.h zfs_prop.h

Add zfs module feature and property info to sysfs

This extends our sysfs '/sys/module/zfs' entry to include feature
and property attributes. The primary consumer of this information
is user processes, like the zfs CLI, that need to know what the
current loaded ZFS module supports. The libzfs binary will consult
this information when instantiating the zfs and zpool property
tables and the pool features table.

This introduces 4 kernel objects (dirs) into '/sys/module/zfs'
with corresponding attributes (files):
  features.runtime
  features.pool
  properties.dataset
  properties.pool

Reviewed-by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Don Brady <don.brady at delphix.com>
Closes #7706

ZFS on Linux — tests/test-runner/bin zts-report.py, tests/zfs-tests/tests/functional/channel_program/synctask_core tst.list_snapshots.ksh tst.list_children.ksh

ZTS: Fix EBUSY volume destroy failures

It's possible for an unrelated process, like blkid, to have the
volume open when 'zfs destroy' is run.  Switch the cleanup functions
to the destroy_dataset() helper which handles this case by retrying
the destroy when the dataset is busy.  This was done not only for
volumes but also for file systems for consistency.

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7854

ZFS on Linux — module/zfs vdev.c

Allow ECKSUM in vdev_checkpoint_sm_object()

The checkpoint space map object may not be accessible from the
vdev's ZAP when it has been damaged.  This may be the case when
performing an extreme rewind when importing the pool.

Reviewed-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Reviewed by: Tim Chase <tim at chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7809
Closes #7853
Delta File
+8 -4 module/zfs/vdev.c
+8 -4 1 file

ZFS on Linux — module/zfs dbuf.c

clean up __dbuf_hold_impl

We can simplify the dbuf_hold code by allocating dbuf_hold_arg_t's on
demand, rather than allocating a big array of them up front.  While this
can occasionally increase the number of allocations, typically only one
allocation is needed since the indirect block is already cached.

The performance test suite gets the same results with this change.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen at delphix.com>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes #7841
Delta File
+37 -43 module/zfs/dbuf.c
+37 -43 1 file

ZFS on Linux — tests/zfs-tests/tests/functional/pool_checkpoint pool_checkpoint.kshlib

ZTS: pool_checkpoint path cleanup

Removing hardcoded paths in pool_checkpoint.kshlib

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7840

ZFS on Linux — tests/zfs-tests/include libtest.shlib

ZTS: Fix DEV_DSKDIR trim from disk

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Richard Elling <Richard.Elling at RichardElling.com>
Closes #7848

ZFS on Linux — tests/zfs-tests/tests/functional/zvol/zvol_swap zvol_swap_003_pos.ksh

ZTS: zvol_swap_003 path cleanup

Removing hardcoded paths in zvol_swap_003

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7839

ZFS on Linux — cmd/arc_summary arc_summary3.py, tests/zfs-tests/tests/functional/cli_root/zdb zdb_002_pos.ksh

ZTS: path cleanup

Removing hardcoded paths in many scripts.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bernie1995 <bernie.pikes at gmail.com>
Issue #7507
Closes #7843

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zfs_create zfs_create_013_pos.ksh

ZTS: Fix zfs_create_013_pos

It's possible for an unrelated process, like blkid, to have the
volume open when 'zfs destroy' is run.  Switch the cleanup function
to the destroy_dataset() helper which handles this case by retrying
the destroy when the dataset is busy.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7847

ZFS on Linux — cmd/zinject zinject.c, cmd/ztest ztest.c

OpenZFS 9403 - assertion failed in arc_buf_destroy()

Assertion failed in arc_buf_destroy() when concurrently reading
block with checksum error.

Porting notes:
* The ability to zinject decompression errors has been added, but
  this only works at the zio_decompress() level, where we have all
  of the info we need to match against the user's zinject options.
* The decompress_fault test has been added to test the new zinject
  functionality
* We attempted to set zio_decompress_fail_fraction to (1 << 18) in
  ztest for further test coverage. Although this did uncover a few
  low priority issues, this unfortuantely also causes ztest to
  ASSERT in many locations where the code is working correctly since
  it is designed to fail on IO errors. Developers can manually set
  this variable with the '-o' option to find and debug issues.

Authored by: Matt Ahrens <mahrens at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed by: Paul Dagnelie <pcd at delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Approved by: Matt Ahrens <mahrens at delphix.com>
Ported-by: Tom Caputi <tcaputi at datto.com>

    [4 lines not shown]

ZFS on Linux — . configure.ac, config kernel-vfs-direct_IO.m4 kernel.m4

Direct IO support

Direct IO via the O_DIRECT flag was originally introduced in XFS by
IRIX for database workloads. Its purpose was to allow the database
to bypass the page and buffer caches to prevent unnecessary IO
operations (e.g.  readahead) while preventing contention for system
memory between the database and kernel caches.

On Illumos, there is a library function called directio(3C) that
allows user space to provide a hint to the file system that Direct IO
is useful, but the file system is free to ignore it. The semantics
are also entirely a file system decision. Those that do not
implement it return ENOTTY.

Since the semantics were never defined in any standard, O_DIRECT is
implemented such that it conforms to the behavior described in the
Linux open(2) man page as follows.

    1.  Minimize cache effects of the I/O.

    By design the ARC is already scan-resistant which helps mitigate
    the need for special O_DIRECT handling.  Data which is only
    accessed once will be the first to be evicted from the cache.
    This behavior is in consistent with Illumos and FreeBSD.

    [47 lines not shown]

ZFS on Linux — rpm/generic zfs-kmod.spec.in zfs.spec.in

Remove %changelog from spec file

Remove the %changelog section from the spec files since it does
not get updated in the master branch.  Not only does this mean
the information is stale, but it can result in 'make deb' failing
to build packages, issue #7825.  This section should be updated
for tagged releases.

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7825
Closes #7827

ZFS on Linux — lib/libshare smb.c, module/icp/core kcf_mech_tabs.c

Fedora 28: Fix misc bounds check compiler warnings

Fix a bunch of truncation compiler warnings that show up
on Fedora 28 (GCC 8.0.1).

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Issue #7368
Closes #7826
Closes #7830

ZFS on Linux — rpm/generic zfs.spec.in

Fix libaio-devel requirement for Debian-based distributions

BuildRequires tags for "-devel" packages in the RPM spec file do not
work when building on Debian-based distributions.

Fix this issue by making this requirement conditional to RPM-based
distributions.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7829
Closes #7831
Delta File
+2 -0 rpm/generic/zfs.spec.in
+2 -0 1 file

ZFS on Linux — rpm/generic zfs.spec.in

Add libaio-devel BuildRequires

The zfs-test package needs a build requirement on the libaio-devel
package.  Without it ./configure will correctly determine that
mmap_libaio cannot be built and it will be skipped.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7821
Closes #7824
Delta File
+1 -0 rpm/generic/zfs.spec.in
+1 -0 1 file

ZFS on Linux — module/zfs dsl_destroy.c, tests/runfiles linux.run

Stack overflow when destroying deeply nested clones

Destroy operations on deeply nested chains of clones can overflow
the stack:

        Depth    Size   Location    (221 entries)
        -----    ----   --------
  0)    15664      48   mutex_lock+0x5/0x30
  1)    15616       8   mutex_lock+0x5/0x30
...
 26)    13576      72   dsl_dataset_remove_clones_key.isra.4+0x124/0x1e0 [zfs]
 27)    13504      72   dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs]
 28)    13432      72   dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs]
...
185)     2128      72   dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs]
186)     2056      72   dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs]
187)     1984      72   dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs]
188)     1912     136   dsl_destroy_snapshot_sync_impl+0x4e0/0x1090 [zfs]
189)     1776      16   dsl_destroy_snapshot_check+0x0/0x90 [zfs]
...
218)      304     128   kthread+0xdf/0x100
219)      176      48   ret_from_fork+0x22/0x40
220)      128     128   kthread+0x0/0x100

Fix this issue by converting dsl_dataset_remove_clones_key() from

    [7 lines not shown]

ZFS on Linux — cmd/arc_summary arc_summary.py arc_summary3.py

Added metadata/dnode cache info to arc_summary

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rich Ercolani <rincebrain at gmail.com>
Closes #7815

ZFS on Linux — module/zfs vdev.c

s/VERIFY/VERIFY3S in vdev_checkpoint_sm_object

Using VERIFY3S allows to view the unexpected error value in the system
log.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Don Brady <don.brady at delphix.com>
Signed-off-by: Tim Chase <tim at chase2k.com>
Issue #7809
Closes #7818
Delta File
+2 -1 module/zfs/vdev.c
+2 -1 1 file

ZFS on Linux — include/sys dsl_dataset.h, module/zfs dsl_dataset.c zfs_vfsops.c

Always wait for txg sync when umounting dataset

Currently, when unmounting a filesystem, ZFS will only wait for
a txg sync if the dataset is dirty and not readonly. However, this
can be problematic in cases where a dataset is remounted readonly
immediately before being unmounted, which often happens when the
system is being shut down. Since encrypted datasets require that
all I/O is completed before the dataset is disowned, this issue
causes problems when write I/Os leak into the txgs after the
dataset is disowned, which can happen when sync=disabled.

While looking into fixes for this issue, it was discovered that
dsl_dataset_is_dirty() does not return B_TRUE when the dataset has
been removed from the txg dirty datasets list, but has not actually
been processed yet. Furthermore, the implementation is comletely
different from dmu_objset_is_dirty(), adding to the confusion.
Rather than relying on this function, this patch forces the umount
code path (and the remount readonly code path) to always perform a
txg sync on read-write datasets and removes the function altogether.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7753
Closes #7795

ZFS on Linux — include/sys txg.h, module/zfs txg.c dmu_tx.c

Small rework of txg_list code

This patch simply adds some missing locking to the txg_list
functions and refactors txg_verify() so that it is only compiled
in for debug builds.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7795
Delta File
+44 -20 module/zfs/txg.c
+7 -0 include/sys/txg.h
+1 -1 module/zfs/dmu_tx.c
+52 -21 3 files

ZFS on Linux — module/zfs arc.c dmu_send.c, tests/runfiles linux.run

Fix issues with raw receive_write_byref()

This patch fixes 2 issues with raw, deduplicated send streams. The
first is that datasets who had been completely received earlier in
the stream were not still marked as raw receives. This caused
problems when newly received datasets attempted to fetch raw data
from these datasets without this flag set.

The second problem was that the arc freeze checksum code was not
consistent about which locks needed to be held while performing
its asserts. The proper locking needed to run these asserts is
actually fairly nuanced, since the asserts touch the linked list
of buffers (requiring the header lock), the arc_state (requiring
the b_evict_lock), and the b_freeze_cksum (requiring the
b_freeze_lock). This seems like a large performance sacrifice and
a lot of unneeded complexity to verify that this relatively small
debug feature is working as intended, so this patch simply removes
these asserts instead.

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7701

ZFS on Linux — contrib/pyzfs/libzfs_core _error_translation.py exceptions.py, contrib/pyzfs/libzfs_core/bindings libzfs_core.py

pyzfs: add missing libzfs_core functions

This change adds the following libzfs_core functions to pyzfs:
lzc_remap, lzc_pool_checkpoint, lzc_pool_checkpoint_discard

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7793
Closes #7800

ZFS on Linux — cmd/zdb zdb.c, tests/runfiles linux.run

Skip import activity test in more zdb code paths

Since zdb opens the pools read-only, it cannot damage the pool in the
event the pool is already imported either on the same host or on
another one.

If the pool vdev structure is changing while zdb is importing the
pool, it may cause zdb to crash.  However this is unlikely, and in any
case it's a user space process and can simply be run again.

For this reason, zdb should disable the multihost activity test on
import that is normally run.

This commit fixes a few zdb code paths where that had been overlooked.
It also adds tests to ensure that several common use cases handle this
properly in the future.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Gu Zheng <guzheng2331314 at 163.com>
Signed-off-by: Olaf Faaland <faaland1 at llnl.gov>
Closes #7797
Closes #7801

ZFS on Linux — cmd/zfs zfs_main.c, cmd/zpool zpool_main.c

Don't modify argv[] in user tools

argv[] gets modified during string parsing for input arguments. This
is reflected in the live process listing. Don't do that.

Reviewed-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: DHE <git at dehacked.net>
Closes #7760
Delta File
+16 -2 cmd/zfs/zfs_main.c
+16 -2 cmd/zpool/zpool_main.c
+32 -4 2 files

ZFS on Linux — include/sys dataset_kstats.h zfs_vfsops.h, include/sys/fs zfs.h

Introduce read/write kstats per dataset

The following patch introduces a few statistics on reads and writes
grouped by dataset. These statistics are implemented as kstats
(backed by aggregate sums for performance) and can be retrieved by
using the dataset objset ID number. The motivation for this change is
to provide some preliminary analytics on dataset usage/performance.

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Signed-off-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Closes #7705
Delta File
+185 -0 module/zfs/dataset_kstats.c
+51 -54 module/zfs/zfs_vnops.c
+59 -0 include/sys/dataset_kstats.h
+33 -16 module/zfs/zvol.c
+10 -1 man/man8/zpool.8
+10 -0 man/man8/zfs.8
+20 -7 10 files not shown
+368 -78 16 files

ZFS on Linux — tests/zfs-tests/tests/functional/events events.cfg

ZTS: events path cleanup

Removing hardcoded paths in events.cfg

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7805

ZFS on Linux — tests/zfs-tests/tests/functional/largest_pool largest_pool_001_pos.ksh

ZTS: largest_pool_001 path cleanup

Removing hardcoded paths in largest_pool_001

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7804

ZFS on Linux — tests/zfs-tests/tests/functional/privilege cleanup.ksh setup.ksh

ZTS: privilege group path cleanup

Removing hardcoded paths in privilege group tests

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7803