Displaying 1 50 of 4,288 commits (0.008s)

ZFS on Linux — lib/libzpool util.c

Fix 'zdb -o' segmentation fault

Fix a regression accidentally introduced by OpenZFS 7280 in ed828c0: since
whether to accept NULL as a valid first parameter in strchr() is implementation
specific we add an additional check to avoid crashing.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #5917
Delta File
+1 -1 lib/libzpool/util.c
+1 -1 1 file

ZFS on Linux — module/zfs zfs_znode.c

Retry zfs_znode_alloc() in zfs_mknode()

For historical reasons zfs_mknode() was written such that it could
never fail.  This poses a problem for Linux since zfs_znode_alloc()
could potentually failure due to low memory.  Handle this gracefully
by retrying zfs_znode_alloc() until it succeeds, direct reclaim
will eventually be able to allocate memory.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5535
Closes #5908
Delta File
+12 -1 module/zfs/zfs_znode.c
+12 -1 1 file

ZFS on Linux — cmd/zed/agents zfs_retire.c zfs_diagnosis.c, lib/libzfs libzfs_fru.c

Fix undefined reference to `libzfs_fru_compare'

Add trivial libzfs_fru_compare() function which can be used when
HAVE_LIBTOPO is not defined.  The only caller is find_vdev() and
this function should never be reached because search_fru must be
NULL unless HAVE_LIBTOPO is defined.

Rename _HAS_FMD_TOPO to existing HAVE_LIBTOPO which was
originally added for this purpose.  This macro will never be defined.

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5402
Closes #5909

ZFS on Linux — cmd/zpool zpool_main.c, lib/libzfs libzfs_pool.c

Fix `zpool status -v` error message

When a pool is suspended it's impossible to read the list
of damaged files from disk.  This would result in a generic
misleading "insufficient permissions" error message.

Update zpool_get_errlog() to use the standard zpool error
logging functions to generate a useful error message.  In
this case:

  errors: List of errors unavailable: pool I/O is currently suspended

This patch does not address the related issue of potentially
not being able to resume a suspend pool when the underlying
device names have changed.

Additionally, remove the error handling from zfs_alloc()
in zpool_get_errlog() for readability since this function
can never fail.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #4031
Closes #5731
Closes #5907

ZFS on Linux — config kernel.m4 zfs-build.m4, module Makefile.in

GZIP compression offloading with QAT accelerator

This patch implement the hardware accelerator method in GZIP compression
in ZFS. When the ZFS pool is enabled GZIP compression, the compression
API will be automatically transferred to the hardware accelerator to
free up CPU resource and speed up the compression time.

* To enable Intel QAT hardware acceleration in ZOL you need to have QAT
  hardware and the driver installed:
  * QAT hardware DH8950:
  http://ark.intel.com/products/79483/Intel-QuickAssist-Adapter-8950
  * QAT driver:
  https://01.org/intel-quickassist-technology
* Start QAT driver in your system:
  service qat_service start
* Enable QAT in ZFS, e.g.:
  ./configure --with-qat=<qat-driver-path>/QAT1.6
  make
* Set GZIP compression in ZFS dataset:
  zfs set compression = gzip <dataset>
* Get QAT hardware statistics by:
  cat /proc/spl/kstat/zfs/qat
* To disable QAT in ZFS:
  insmod zfs.ko zfs_qat_disable=1


    [4 lines not shown]
Delta File
+585 -0 module/zfs/qat_compress.c
+90 -0 config/kernel.m4
+48 -0 module/zfs/qat_compress.h
+17 -0 module/zfs/gzip.c
+3 -0 config/zfs-build.m4
+3 -0 module/zfs/spa_misc.c
+3 -0 2 files not shown
+749 -0 8 files

ZFS on Linux — lib/libspl/include/sys isa_defs.h

libspl: Fix incorrect use of platform defines on sparc64

libspl tries to detect sparc64 by checking whether __sparc64__
is defined. Unfortunately, this assumption is not correct as
sparc64 does not define __sparc64__ but it defines __sparc__
and __arch64__ instead. This leads to sparc64 being detected
as 32-Bit sparc and the build fails because both _ILP32 and
_LP64 are defined in this case.

To fix the problem, remove the checks for __sparc64__ and
just check __arch64__ if a sparc host was previously
detected with __sparc__.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: John Paul Adrian Glaubitz <glaubitz at physik.fu-berlin.de>
Closes #5913
Delta File
+2 -2 lib/libspl/include/sys/isa_defs.h
+2 -2 1 file

ZFS on Linux — include/sys dmu_objset.h dnode.h, module/zfs dmu_objset.c arc.c

OpenZFS 7968 - multi-threaded spa_sync()

Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed by: Brad Lewis <brad.lewis at delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov at nexenta.com>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Ported-by: Matthew Ahrens <mahrens at delphix.com>

spa_sync() iterates over all the dirty dnodes and processes each of them
by calling dnode_sync(). If there are many dirty dnodes (e.g. because we
created or removed a lot of files), the single thread of spa_sync()
calling dnode_sync() can become a bottleneck. Additionally, if many
dnodes are dirtied concurrently in open context (e.g. due to concurrent
file creation), the os_lock will experience lock contention via
dnode_setdirty().

The solution is to track dirty dnodes on a multilist_t, and for
spa_sync() to use separate threads to process each of the sublists in
the multilist.

OpenZFS-issue: https://www.illumos.org/issues/7968
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/4a2a54c
Closes #5752
Delta File
+153 -35 module/zfs/dmu_objset.c
+43 -43 module/zfs/arc.c
+26 -12 module/zfs/zio.c
+7 -29 module/zfs/dnode.c
+23 -5 module/zfs/dsl_pool.c
+17 -10 module/zfs/multilist.c
+34 -25 10 files not shown
+303 -159 16 files

ZFS on Linux — config kernel-inode-getattr.m4 kernel.m4, include/linux vfs_compat.h

Linux 4.11 compat: iops.getattr and friends

In torvalds/linux at a528d35, there are changes to the getattr family of functions,
struct kstat, and the interface of inode_operations .getattr.

The inode_operations .getattr and simple_getattr() interface changed to:

int (*getattr) (const struct path *, struct dentry *, struct kstat *,
    u32 request_mask, unsigned int query_flags)

The request_mask argument indicates which field(s) the caller intends to use.
Fields the caller has not specified via request_mask may be set in the returned
struct anyway, but their values may be approximate.

The query_flags argument indicates whether the filesystem must update
the attributes from the backing store.

Currently both fields are ignored.  It is possible that getattr-related
functions within zfs could be optimized based on the request_mask.

struct kstat includes new fields:
u32               result_mask;  /* What fields the user got */
u64               attributes;   /* See STATX_ATTR_* flags */
struct timespec   btime;        /* File creation time */


    [8 lines not shown]

ZFS on Linux — man/man8 zfs.8

zfs(8) fixes

Documentation fixes for zfs(8)

* White space issue in the userused at user property section
* zfs send supports using bookmarks as the origin snapshot

Reviewed by: Ned Bass <bass6 at llnl.gov>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: DHE <git at dehacked.net>
Closes #5906
Delta File
+8 -7 man/man8/zfs.8
+8 -7 1 file

ZFS on Linux — module/zfs dmu_tx.c

OpenZFS 7801 - add more by-dnode routines (lint)

Authored by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Ported-by: Giuseppe Di Natale <dinatale2 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/7801
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/f25efb3
Closes #5894
Delta File
+3 -3 module/zfs/dmu_tx.c
+3 -3 1 file

ZFS on Linux — include/sys zfs_vfsops.h, module/zfs zfs_vfsops.c zpl_super.c

Align mount options handling and type/function names with OpenZFS

Refactor the temporary mount option in a way which minimizes
differences with upstream.  Additionally, replace the zfs_sb_t
type with zfsvfs_t and rename several functions to be consistent
with the upstream names.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5876
∈ Brian Behlendorf - 1c2555ef - 2017-03-09 00:56:09
Restructure mount option handling

Restructure the handling of mount options to be consistent with
upstream OpenZFS.  This required making the following changes.

- The zfs_mntopts_t was renamed vfs_t and adjusted to provide
  the minimal needed functionality.  This includes a pointer
  back to the associated zfsvfs_t.  Plus it made it possible
  to revert zfs_register_callbacks() and zfsvfs_create() back
  to their original prototypes.

- A zfs_mnt_t structure was added for the sole purpose of
  providing a structure to pass the osname and raw mount
  pointer to zfs_domount() without having to copy them.

- Mount option parsing was moved down from the zpl_* wrapper
  functions in to the zfs_* functions.  This allowed for the
  code to be simplied and it's where similar functionality
  appears on other platforms.

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>

ZFS on Linux — module/zfs zvol.c

Fix ZVOL BLKFLSBUF ioctl

The BLKFLSBUF ioctl is expected to do two things:

  - flush dirty pages to stable storage, and
  - invalidate clean pages

Unfortunately, the existing implementation of BLKFLSBUF in
zvol_ioctl() only flushes pages which are part of the current
TXG to disk.  There may be additional dirty pages in the
page cache which haven't yet been submitted to the DMU and
therefore aren't part of any TXG.

Furthermore because zvol_ioctl() returns 0 the generic
blkdev_flushbuf() does not invalidate the page cache.

Resolve the issue by moving bdev_flush() in to zvol_ioctl()
and explicitly waiting for a full TXG sync.  Then invalidate
the page cache.  The associated ARC buffers need not be
evicted since they cannot be bypassed using O_DIRECT.

Reviewed-by: Chunwei Chen <david.chen at osnexus.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5871
Closes #5879
Delta File
+12 -3 module/zfs/zvol.c
+12 -3 1 file

ZFS on Linux — . Makefile.am, module/zfs zfs_vnops.c

Suppress cppcheck nullPointer error in zfs_write

Newer versions of cppcheck find the potential NULL pointer
bug in zfs_write(). The function is difficult to refactor without
extensive work, so suppress the potential NULL pointer error
which cannot occur for now.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #5882
Delta File
+2 -1 Makefile.am
+1 -0 module/zfs/zfs_vnops.c
+3 -1 2 files

ZFS on Linux — cmd/arc_summary arc_summary.py, cmd/dbufstat dbufstat.py

Correct arc_summary and dbufstat python style

arc_summary and dbufstat should have two spaces
after their last function definitions.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #5881

ZFS on Linux — . Makefile.am, cmd/zed/zed.d statechange-led.sh zed-functions.sh

Enable shellcheck to run for select scripts

Enable shellcheck to run on zed scripts,
paxcheck.sh, zfs-tests.sh, zfs.sh, and zloop.sh.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #5812
Delta File
+35 -30 scripts/zfs-tests.sh
+30 -16 cmd/zed/zed.d/statechange-led.sh
+26 -19 scripts/zloop.sh
+0 -17 scripts/common.sh.in
+5 -7 scripts/zfs.sh
+7 -4 cmd/zed/zed.d/zed-functions.sh
+11 -7 3 files not shown
+114 -100 9 files

ZFS on Linux — cmd/ztest ztest.c, include/sys zfs_vfsops.h zfs_vnops.h

Rename zfs_* functions

Several functions were renamed when ZFS was originally ported to
Linux.  Revert the code to the original names to minimize the
delta with upstream OpenZFS.

  zfs_sb_teardown -> zfsvfs_teardown
  zfs_sb_create -> zfsvfs_create
  zfs_sb_setup -> zfsvfs_setup
  zfs_sb_free -> zfsvfs_free
  get_zfs_sb -> getzfsvfs
  zfs_sb_hold -> zfsvfs_hold
  zfs_sb_rele -> zfsvfs_rele

  zfs_sb_prune_aliases  -> zfs_prune_aliases (Linux-only)
  zfs_sb_prune -> zfs_prune (Linux only)

Align the zfs_vnops.h and zfs_vfsops.h with upstream as much
as possible.  Several prototypes were removed and those that
remain were reordered.

Move the EXPORT_SYMBOL lines to the end of the source files
for consistency with the other source files.

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>

ZFS on Linux — include/sys zfs_ctldir.h, lib/libshare nfs.c

Fix nfs snapdir automount

The current implementation for allowing nfs to access snapdir is very buggy.
It uses a special fh for snapdirs, such that the next time nfsd does
fh_to_dentry, it actually returns the root inode inside the snapshot. So nfsd
never knows it cross a mountpoint.

The problem is that nfsd will not hold a reference on the vfsmount of the
snapshot. This cause auto unmounter to unmount the snapshot even though nfs is
still holding dentries in it.

To fix this, we return the inode for the snapdirs themselves. However, we also
trigger automount upon fh_to_dentry, and return ESTALE so nfsd will revalidate
and see the mountpoint and do crossmnt.

Because nfsd will now be aware that these are different filesystems users
must add crossmnt to their export options to access snapshot directories.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Chunwei Chen <david.chen at osnexus.com>
Closes #3794
Closes #4716
Closes #5810
Closes #5833

ZFS on Linux — include/linux blkdev_compat.h

Fix harmless "BARRIER is deprecated" kernel warning on Centos 6.8

A one time warning after module load that "BARRIER is deprecated" was seen
on the heavily patched 2.6.32-642.13.1.el6.x86_64 Centos 6.8 kernel.  It seems
that kernel had both the old BARRIER and the newer FLUSH/FUA interfaces
defined.  This fixes the warning by prefering the newer FLUSH/FUA interface
if it's available.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #5739
Closes #5828
Delta File
+6 -9 include/linux/blkdev_compat.h
+6 -9 1 file

ZFS on Linux — include/sys zfs_vfsops.h zfs_znode.h, module/zfs zfs_vnops.c zfs_vfsops.c

Rename zfs_sb_t -> zfsvfs_t

The use of zfs_sb_t instead of zfsvfs_t results in unnecessary
conflicts with the upstream source.  Change all instances of
zfs_sb_t to zfsvfs_t including updating the variables names.

Whenever possible the code was updated to be consistent with
hope it appears in the upstream OpenZFS source.

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Delta File
+357 -353 module/zfs/zfs_vnops.c
+351 -348 module/zfs/zfs_vfsops.c
+179 -177 module/zfs/zfs_znode.c
+82 -81 module/zfs/zfs_fuid.c
+78 -78 module/zfs/zfs_dir.c
+77 -77 module/zfs/zfs_ioctl.c
+364 -362 16 files not shown
+1,488 -1,476 22 files

ZFS on Linux — man/man8 zpool.8

Corrected highlight for zpool man page

SS is already highlighted and the fB/fR tags break the highlighting
prematurely, removing the tags highlights the entire line.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #5873
Delta File
+1 -1 man/man8/zpool.8
+1 -1 1 file

ZFS on Linux — config kernel-objtool.m4 kernel.m4, module/icp Makefile.in

[icp] fpu and asm cleanup for linux

Properly annotate functions and data section so that objtool does not complain
when CONFIG_STACK_VALIDATION and CONFIG_FRAME_POINTER are enabled.

Pass KERNELCPPFLAGS to assembler.

Use kfpu_begin()/kfpu_end() to protect SIMD regions in Linux kernel.

Reviewed-by: Tom Caputi <tcaputi at datto.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Gvozden Neskovic <neskovic at gmail.com>
Closes #5872
Closes #5041

ZFS on Linux — include/linux blkdev_compat.h

Fix multi-line error messages in blkdev_compat.h

Fix multi-line error messages in blkdev_compat.h by changing
error-generating multi-line error messages to single line errors.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #5860
Delta File
+4 -9 include/linux/blkdev_compat.h
+4 -9 1 file

ZFS on Linux — . configure.ac, config zfs-build.m4

OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space

Reviewed by: Steve Gonczi <steve.gonczi at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

Background information: This assertion about tx_space_* verifies that we
are not dirtying more stuff than we thought we would. We “need” to know
how much we will dirty so that we can check if we should fail this
transaction with ENOSPC/EDQUOT, in dmu_tx_assign(). While the
transaction is open (i.e. between dmu_tx_assign() and dmu_tx_commit() —
typically less than a millisecond), we call dbuf_dirty() on the exact
blocks that will be modified. Once this happens, the temporary
accounting in tx_space_* is unnecessary, because we know exactly what
blocks are newly dirtied; we call dnode_willuse_space() to track this
more exact accounting.

The fundamental problem causing this bug is that dmu_tx_hold_*() relies
on the current state in the DMU (e.g. dn_nlevels) to predict how much
will be dirtied by this transaction, but this state can change before we
actually perform the transaction (i.e. call dbuf_dirty()).

This bug will be fixed by removing the assertion that the tx_space_*
accounting is perfectly accurate (i.e. we never dirty more than was

    [31 lines not shown]
Delta File
+176 -669 module/zfs/dmu_tx.c
+9 -75 module/zfs/dbuf.c
+0 -83 module/zfs/zap_micro.c
+0 -61 module/zfs/zap.c
+18 -22 module/zfs/dsl_dir.c
+0 -38 module/zfs/dsl_dataset.c
+29 -111 16 files not shown
+232 -1,059 22 files

ZFS on Linux — module/zfs dsl_dataset.c

OpenZFS 7843 - get_clones_stat() is suboptimal for lots of clones

Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/7843
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/4d519e7
Closes #5868
Delta File
+12 -1 module/zfs/dsl_dataset.c
+12 -1 1 file

ZFS on Linux — cmd/zdb zdb.c, man/man8 zdb.8

Dump unique configurations and Uberblocks in zdb -lu

For zdb -l, detect when the configuration nvlist in some label l (l>0)
is the same as a configuration already dumped.  If so, do not dump it.

Make a similar check when dumping Uberblocks for zdb -lu.  Check whether
a label already dumped contains an identical Uberblock.  If so, do not
dump the Uberblock.

When dumping a configuration or Uberblock, state which labels it is
found in (0-3), for example: labels = 1 2 3

Detecting redundant uberblocks or configurations is accomplished by
calculating checksums of the uberblocks and the packed nvlists
containing the configuration.

If there is nothing unique to be dumped for a label (ie the
configuration and uberblocks have checksums matching those already
dumped) print nothing for that label.

With additional l's or u's, increase verbosity as follows:

-l      Dump each unique configuration only once.
        Indicate which labels it appears in.
-ll     In addition, dump label space usage stats.

    [66 lines not shown]

ZFS on Linux — config kernel-vfs-rw-iterate.m4, module/zfs zpl_file.c

Fix loop device becomes read-only

Commit 933ec99 removes read and write from f_op because the vfs layer will
select iter_write or aio_write automatically. However, for Linux <= 4.0,
loop_set_fd will actually check f_op->write and set read-only if not exists.
This patch add them back and use the generic do_sync_{read,write} for
aio_{read,write} and new_sync_{read,write} for {read,write}_iter.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Chunwei Chen <david.chen at osnexus.com>
Closes #5776
Closes #5855

ZFS on Linux — lib/libspl/include/sys isa_defs.h, rpm/generic zfs-kmod.spec.in

Fix powerpc build

Unlike other architectures which sanitize the LDFLAGS from the
environment in arch/<arch>/Makefile.  The powerpc Makefile
allows LDFLAGS to be passed through resulting in the following
build failure.

  /usr/bin/ld: unrecognized option '-Wl,-z,relro'

LDFLAGS is set in /usr/lib/rpm/redhat/macros by default.  Clear
the environment variable when building kmods for powerpc.

Additionally, now that ppc64le exists it's not longer safe to
assume a powerpc system is big endian.  Rely on the endianness
provided by the compiler.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5856

ZFS on Linux — tests/zfs-tests/tests/functional/zvol zvol_common.shlib zvol.cfg, tests/zfs-tests/tests/functional/zvol/zvol_ENOSPC setup.ksh

Reduce size of zvol and enforce 4k blocksize in zvol tests

32-bit builders in the buildbot are having trouble completing
their ENOSPC testing in less than the timeout. Reduce the
zvol size and use a 4k block size to reduce read-modify-writes
which are particularly expensive on 32-bit systems due to the
reduced maximum ARC size.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Kash Pande <kash at tripleback.net>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #5845

ZFS on Linux — scripts zfs-tests.sh

Bug fixes for single test runs in zfs-tests

Correctly remove the temporary runfile after the
single test is run.

Cleanup and setup scripts are relative to the
test suite's location, correct how we look for
those scripts.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #5844
Delta File
+5 -5 scripts/zfs-tests.sh
+5 -5 1 file

ZFS on Linux — include/sys refcount.h, module/zfs refcount.c

Linux 4.11 compat: avoid refcount_t name conflict

Linux 4.11 introduces a new type, refcount_t, which conflicts with the
type of the same name defined within ZFS.

Rename the ZFS type zfs_refcount_t.  Within the ZFS code, use a macro to
cause references to refcount_t to be changed to zfs_refcount_t at
compile time.  This reduces conflicts when later landing OpenZFS
patches.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Olaf Faaland <faaland1 at llnl.gov>
Closes #5823
Closes #5842
Delta File
+14 -3 include/sys/refcount.h
+1 -1 module/zfs/refcount.c
+15 -4 2 files

ZFS on Linux — module/zfs arc.c

OpenZFS 7867 - ARC space accounting leak

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Tim Chase <tim at chase2k.com>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/7867
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/aa1f740d
Closes #5874
Delta File
+6 -0 module/zfs/arc.c
+6 -0 1 file

ZFS on Linux — contrib/initramfs/hooks zfs

Fix initramfs hook for merged /usr/lib and /lib

Under a merged `/lib` -> `/usr/lib` which renders `/lib` as a symlink,
`find /lib -type f -name libgcc_s.so.1` will not return a result as
`find` will not traverse the symlink. Modifying it to `find /lib/ -type
f -name libgcc_s.so.1` should work for both symlinked and non-symlinked
`/lib` directories.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matt Kemp <matt at mattikus.com>
Closes #5834
Delta File
+1 -1 contrib/initramfs/hooks/zfs
+1 -1 1 file

ZFS on Linux — include/sys dmu_tx.h, module/zfs dmu_tx.c dmu_object.c

Clean up by-dnode code in dmu_tx.c

https://github.com/zfsonlinux/zfs/commit/0eef1bde31d67091d3deed23fe2394f5a8bf2276
introduced some changes which we slightly improved the style of when
porting to illumos.

There is also one minor error-handling fix, in zap_add() the "zap" may
become NULL in case of an error re-opening the ZAP.

Originally suggested at: https://github.com/openzfs/openzfs/pull/276

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes #5805

ZFS on Linux — module/zfs vdev_raidz.c

ABD style cleanups

The commit a6255b7fce400d485a0e87cbe369aa0ed7dc5dc4 removed a few
assertions which help catch errors and improve code readability. It also
duplicated two conditionals, which was unnecessary and made the code
confusing to read. This patch cleans it up.

Reviewed-by: David Quigley <david.quigley at intel.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Isaac Huang <he.huang at intel.com>
Closes #5802
Delta File
+13 -12 module/zfs/vdev_raidz.c
+13 -12 1 file

ZFS on Linux — lib/libzfs libzfs_sendrecv.c

Fix checksumflags assignment in cksummer

drr_checksumflags was incorrectly set to drr_checksumtype.

Reviewed-by: Matt Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Tim Crawford <tcrawford at datto.com>
Closes #5830
Delta File
+1 -1 lib/libzfs/libzfs_sendrecv.c
+1 -1 1 file

ZFS on Linux — scripts zfs-tests.sh

Allow zfs-tests to run a single test

Add a -t flag to zfs-tests to allow a user
to run a single test by providing the path
to the test relative to STF_SUITE.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #5775
Delta File
+63 -1 scripts/zfs-tests.sh
+63 -1 1 file

ZFS on Linux — module/zfs spa.c

Fix incorrect spare vdev state after replacing

After a hot spare replaces an OFFLINE vdev, the new
parent spare vdev state is set incorrectly to OFFLINE.
The correct state should be DEGRADED. The incorrect
OFFLINE state will prevent top-level vdev from reading
the spare vdev, thus causing unnecessary reconstruction.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Don Brady <don.brady at intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Isaac Huang <he.huang at intel.com>
Closes #5766
Closes #5770
Delta File
+5 -0 module/zfs/spa.c
+5 -0 1 file

ZFS on Linux — module/nvpair nvpair.c, module/zfs zfs_dir.c zio.c

OpenZFS 7812 - Remove gender specific language

Authored by: Daniel Hoffman <dj.hoffman at delphix.com>
Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed by: Prakash Surya <prakash.surya at delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi at delphix.com>
Reviewed by: Chris Williamson <chris.williamson at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed by: Igor Kozhukhov <igor at dilos.org>
Reviewed by: Dan McDonald <danmcd at omniti.com>
Reviewed by: Robert Mustacchi <rm at joyent.com>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Ported-by: George Melikov <mail at gmelikov.ru>

This change removes all gendered language that did not refer specifically
to an individual person or pet. The convention taken was to use
variations on "they" when referring to users and/or human beings, while
using "it" when referring to code, functions, and/or libraries.
Additionally, we took the liberty to fix up any whitespace issues that
were found in any files that were already being modified.

OpenZFS-issue: https://www.illumos.org/issues/7812
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ad626db
Closes #5822

ZFS on Linux — cmd/zed/zed.d statechange-led.sh

Retry setting LED

If the LED is being accessed by another process when we try to update
it, the update will be lost. Add a retry loop which will read the state
of the LED and update it until the LED is in the correct state. The
number of times this will occur is limited to ensure that the ZEDlet
won't hang ZED.

Refactor to remove duplication so setting of the LED occurs in only one
place.

Cleanup a couple of the warnings generated by shellcheck which weren't
the result of specific choices by the author. Several notes and warnings
are still present but removing them would make the code less clear or
require adding lines to tell shellcheck to ignore the warning.

Remove ",i" from the documentation at the top of the file which appears
to be a typographic error.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Christopher Voltz <christopher.voltz at hpe.com>
Closes #5795
Delta File
+33 -55 cmd/zed/zed.d/statechange-led.sh
+33 -55 1 file

ZFS on Linux — include/sys multilist.h arc.h, man/man5 zfs-module-parameters.5

zfs_arc_num_sublists_per_state should be common to all multilists

The global tunable zfs_arc_num_sublists_per_state is used by the ARC and
the dbuf cache, and other users are planned. We should change this
tunable to be common to all multilists.  This tuning may be overridden
on a per-multilist basis.

Reviewed-by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed-by: Dan Kimmel <dan.kimmel at delphix.com>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes #5764

ZFS on Linux — cmd/zed/zed.d README

Fix broken URL

Google moved their style guides to GitHub. Update the shell style guide
URL to the new location.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christopher Voltz <christopher.voltz at hpe.com>
Closes #5797
Delta File
+1 -1 cmd/zed/zed.d/README
+1 -1 1 file

ZFS on Linux — tests/runfiles linux.run, tests/zfs-tests/include default.cfg.in libtest.shlib

Use file-based pools for zpool_expand test 002 and enable it

Use -pH flags in get_pool_prop so that numeric properties such as size
can be compared.  The zpool_expand test suite is currently the only one
which uses get_pool_prop for a numeric property.

Add TEMPFILE and TEMPFILE{0,1,2} to default.cfg for tests that must
build pools on top of files, such as this one where expansion is
necessary but the entries in DISKS may not point to entities that can be
expanded.

Base the pool used for testing on file-type VDEVs instead of using zvols
within an underlying pool, to avoid issues that come up when pools are
backed by other pools.

Remove shell variables EX_1GB and EX_2GB used to recognize correct expansion,
and instead calculate the appropriate values based on the variables used to
control file or volume size, org_size and exp_size.  This change is also
made in test 001 although that test is not enabled because it depends on
FMA.

Finally, enable zpool_expand_002_pos.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>

    [4 lines not shown]

ZFS on Linux — tests/runfiles linux.run

Disable racy snapshot_008_pos

Sometimes zfstests check freed space just after
`zfs destroy snapshot` and get wrong output, because the
space being freed asynchronously in the background.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Issue #5740
Issue #5784
Closes #5785
Delta File
+2 -1 tests/runfiles/linux.run
+2 -1 1 file

ZFS on Linux — module/zfs arc.c

Fix zfs_compressed_arc_enabled parameter description

A likely cut/paste error caused the description to be applied to
zfs_arc_average_blocksize.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tim Chase <tim at chase2k.com>
Closes #5788
Delta File
+1 -1 module/zfs/arc.c
+1 -1 1 file

ZFS on Linux — module/zfs zpl_inode.c

Fix off by one in zpl_lookup

Doing the following command would return success with zfs creating an orphan
object.

        touch $(for i in $(seq 256); do printf "n"; done)

The funny thing is that this will only work once for each directory, because
after upgraded to fzap, zfs_lookup would fail properly since it has additional
length check.

Signed-off-by: Chunwei Chen <david.chen at osnexus.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5768
Delta File
+1 -1 module/zfs/zpl_inode.c
+1 -1 1 file

ZFS on Linux — cmd/zed Makefile.am, cmd/zed/zed.d statechange-led.sh zed-functions.sh

Enclosure LED fixes

- Pass $VDEV_ENC_SYSFS_PATH to 'zpool [iostat|status] -c' to include
  enclosure LED sysfs path.

- Set LEDs correctly after import.  This includes clearing any erroniously
  set LEDs prior to the import, and setting the LED for any UNAVAIL drives.

- Include symlink for vdev_attach-led.sh in Makefile.am.

- Print the VDEV path in all-syslog.sh, and fix it so the pool GUID actually
  prints.

Reviewed-by: Don Brady <don.brady at intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #5716
Closes #5751

ZFS on Linux — lib/libzfs libzfs_import.c

Clear enclosure sysfs path from VDEV label when sysfs path isn't present

This clears vdev_enc_sysfs_path from the label if the VDEV's
/sys/class/block/<dev>/device/enclosure_device path isn't present.

This is important in the case where a disk that is labeled with
vdev_enc_sysfs_path is pulled out and put into another enclosure.
In that case, it's possible that the old sysfs path would be used to
turn on the fault LED for the disk's old slot postion, assuming the
new slot didn't have a LED sysfs entry.

Reviewed-by: Don Brady <don.brady at intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #5524
Closes #5773
Delta File
+3 -1 lib/libzfs/libzfs_import.c
+3 -1 1 file

ZFS on Linux — module/zfs dsl_deadlist.c bpobj.c

panic in bpobj_space(): null pointer dereference

This is a race condition in the deadlist code.

A thread executing an administrative command that uses
dsl_deadlist_space_range() holds the lock of the whole deadlist_t to
protect the access of all its entries that the deadlist contains in an
avl tree.

Sync threads trying to insert a new entry in the deadlist (through
dsl_deadlist_insert() -> dle_enqueue()) do not hold the deadlist lock at
that moment.  If the dle_bpobj is the empty bpobj (our sentinel value),
we close and reopen it.  Between these two operations, it is possible
for the dsl_deadlist_space_range() thread to dereference that bpobj
which is NULL during that window.

Threads should hold the a deadlist's dl_lock when they manipulate its
internal data so scenarios like the one above are avoided.

Reviewed-by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Dan Kimmel <dan.kimmel at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes #5762
Delta File
+23 -9 module/zfs/dsl_deadlist.c
+2 -2 module/zfs/bpobj.c
+25 -11 2 files

ZFS on Linux — tests/zfs-tests/include libtest.shlib, tests/zfs-tests/tests/functional/cli_root/zfs_set zfs_set_common.kshlib

Add `wait_freeing` helper function to ZTS

Sometimes the ZTS checks freed space just after `zfs destroy snapshot` and
gets an unexpected value because of space being freed asynchronously.
For cases like this add a `wait_freeing` function which blocks until the
pools `freeing` property drops to zero.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #5740

ZFS on Linux — etc/systemd/system zfs-mount.service.in

Fix zfs-mount.service failure on boot

The mount(8) command will helpfully try to resolve any device name
which is passed in.  It does this by applying some simple heuristics
before passing it along to the registered mount helper.

Normally this fine.  However, one of these heuristics is to prepend
the current working directory to the passed device name.  If that
resulting directory name exists mount(8) will perform the mount(2)
system call and never invoke the helper utility.

Since the cwd for systemd when running as the system instance is
the root directory the default mount points created by zfs(8) can
cause a mount failure.

This change avoids the issue by explicitly setting the cwd to
a different path when performing the mount.

Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5719