HardenedBSD/hardenedbsd a906febsys/vm vm_unix.c

HBSD: Resolve merge conflict

Signed-off-by:  Shawn Webb <shawn.webb at hardenedbsd.org>
DeltaFile
+0-5sys/vm/vm_unix.c
+0-51 files

HardenedBSD/hardenedbsd 887d9e4lib/libcasper/services/cap_sysctl cap_sysctl.c cap_sysctl.3, lib/libcasper/services/cap_sysctl/tests sysctl_test.c

Merge remote-tracking branch 'origin/freebsd/current/master' into hardened/current/master

Conflicts:
        sys/vm/vm_unix.c (unresolved)

HardenedBSD/hardenedbsd dd0f9ebsys/cddl/contrib/opensolaris/uts/common/fs/zfs zfs_rlock.c vdev_indirect.c, sys/dev/nctgpio nctgpio.c

Merge remote-tracking branch 'freebsd/stable/12' into hardened/12-stable/master

HardenedBSD/hardenedbsd 878f67bsys/riscv/riscv elf_machdep.c

HBSD: Resolve merge conflict

Keep FreeBSD's ASR disabled for RISC-V.

Signed-off-by:  Shawn Webb <shawn.webb at hardenedbsd.org>

HardenedBSD/hardenedbsd 657ef51share/man/man4 superio.4, share/man/man9 superio.9

Merge remote-tracking branch 'origin/freebsd/12-stable/master' into 
hardened/12-stable/master

Conflicts:
        sys/riscv/riscv/elf_machdep.c (unresolved)

HardenedBSD/hardenedbsd 6357299sys/compat/cloudabi64 Makefile, sys/compat/freebsd32 capabilities.conf

Merge remote-tracking branch 'origin/freebsd/12-stable/master' into 
hardened/12-stable/master

Conflicts:
        sys/i386/ibcs2/ibcs2_proto.h (deleted)
        sys/i386/ibcs2/ibcs2_syscall.h (deleted)
        sys/i386/ibcs2/ibcs2_sysent.c (deleted)

HardenedBSD/hardenedbsd 54de8ebstand/libsa/zfs zfsimpl.c, sys/arm/arm unwind.c

Merge branch 'freebsd/current/master' into hardened/current/master

* freebsd/current/master:
  Properly detect ATA sanitize errors.
  Apply a small optimization to pmap_remove_l3_range().  Specifically, hoist a 
PHYS_TO_VM_PAGE() operation that always returns the same vm_page_t out of the loop.  
(Since arm64 is configured as VM_PHYSSEG_SPARSE, the implementation of PHYS_TO_VM_PAGE() 
is more costly than that of VM_PHYSSEG_DENSE platforms, like amd64.)
  loader: rewrite zfs vdev initialization
  schedlock 4/4
  powerpc/powernv: Set the PTCR for the Nest MMU
  schedlock 3/4
  schedlock 2/4
  Rewrite arm kernel stack unwind code to work when unwinding through modules.
  schedlock 1/4
  powerpc/mpc85xx: Clean up Freescale SATA driver a little
  Support --all-repeats in uniq(1) for compatibility with gnu coreutils.

HardenedBSD/hardenedbsd d0889a1sys/contrib/ipfilter/netinet fil.c

Merge branch 'freebsd/12-stable/master' into hardened/12-stable/master

* freebsd/12-stable/master:
  MFC r355669:

HardenedBSD/hardenedbsd d4bb55bsbin/camcontrol camcontrol.c

Properly detect ATA sanitize errors.

It seems I read specifications not careful enough.  There are devices not
setting successful completion bit, causing previous code report false error.

MFC after:      1 week

HardenedBSD/hardenedbsd 21262f8sys/arm64/arm64 pmap.c

Apply a small optimization to pmap_remove_l3_range().  Specifically, hoist a
PHYS_TO_VM_PAGE() operation that always returns the same vm_page_t out of
the loop.  (Since arm64 is configured as VM_PHYSSEG_SPARSE, the
implementation of PHYS_TO_VM_PAGE() is more costly than that of
VM_PHYSSEG_DENSE platforms, like amd64.)

MFC after:      1 week

HardenedBSD/hardenedbsd c29918astand/libsa/zfs zfsimpl.c, sys/cddl/boot/zfs zfsimpl.h zfssubr.c

loader: rewrite zfs vdev initialization

In some cases the pool discovery will get stuck in infinite loop while setting
up the vdev children.

To fix, we split the vdev setup into two parts, first we create vdevs based on
configuration we do get from pool label, then, we process pool config from MOS
and update the pool config if needed.

Testing done: confirm previously hung loader is not hung any more.

MFC after:      1 week

HardenedBSD/hardenedbsd d828c22sys/contrib/ipfilter/netinet fil.c

MFC r355669:

in6_cksum() returns zero when checksums are good.

HardenedBSD/hardenedbsd f62435bsys/contrib/ipfilter/netinet fil.c

MFC r355669:

in6_cksum() returns zero when checksums are good.

HardenedBSD/hardenedbsd 506c867sys/kern sched_ule.c sched_4bsd.c

schedlock 4/4

Don't hold the scheduler lock while doing context switches.  Instead we
unlock after selecting the new thread and switch within a spinlock
section leaving interrupts and preemption disabled to prevent local
concurrency.  This means that mi_switch() is entered with the thread
locked but returns without.  This dramatically simplifies scheduler
locking because we will not hold the schedlock while spinning on
blocked lock in switch.

This change has not been made to 4BSD but in principle it would be
more straightforward.

Discussed with: markj
Reviewed by:    kib
Tested by:      pho
Differential Revision: https://reviews.freebsd.org/D22778

HardenedBSD/hardenedbsd de75c61sys/powerpc/powernv platform_powernv.c opal.h

powerpc/powernv: Set the PTCR for the Nest MMU

The Nest MMU manages address translation for accelerators on the POWER9.  To
do so, it needs a page table, so export the system page table to the Nest
MMU.  This will quietly fail on pre-POWER9 systems that do not have a NMMU.

The NMMU is currently unused, so this change is currently effectively a NOP,
but the NMMU and VAS will eventually be used.

HardenedBSD/hardenedbsd d71b815sys/kern subr_turnstile.c

schedlock 3/4

Eliminate lock recursion from turnstiles.  This was simply used to avoid
tracking the top-level turnstile lock.  explicitly check for it before
picking up and dropping locks.

Reviewed by:    kib
Tested by:      pho
Differential Revision:  https://reviews.freebsd.org/D22746

HardenedBSD/hardenedbsd 7431380sys/kern subr_sleepqueue.c, sys/sys proc.h

schedlock 2/4

Do all sleepqueue post-processing in sleepq_remove_thread() so that we
do not require the thread lock after a context switch.

Reviewed by:    jhb, kib
Differential Revision:  https://reviews.freebsd.org/D22745

HardenedBSD/hardenedbsd 80d5f06sys/arm/arm unwind.c elf_machdep.c, sys/arm/include stack.h

Rewrite arm kernel stack unwind code to work when unwinding through modules.

The arm kernel stack unwinder has apparently never been able to unwind when
the path of execution leads through a kernel module. There was code that
tried to handle modules by looking for the unwind data in them, but it did
so by trying to find symbols which have never existed in arm kernel
modules. That caused the unwind code to panic, and because part of panic
handling calls into the unwind code, that just created a recursion loop.

Locating the unwind data in a loaded module requires accessing the Elf
section headers to find the SHT_ARM_EXIDX section. For preloaded modules
those headers are present in a metadata blob. For dynamically loaded
modules, the headers are present only while the loading is in progress; the
memory is freed once the module is ready to use. For that reason, there is
new code in kern/link_elf.c, wrapped in #ifdef __arm__, to extract the
unwind info while the headers are loaded. The values are saved into new
fields in the linker_file structure which are also conditional on __arm__.

In arm/unwind.c there is new code to locally cache the per-module info
needed to find the unwind tables. The local cache is crafted for lockless
read access, because the unwind code often needs to run in context where
sleeping is not allowed.  A large comment block describes the local cache
list, so I won't repeat it all here.

HardenedBSD/hardenedbsd bf925a1sys/kern subr_sleepqueue.c sched_ule.c

schedlock 1/4

Eliminate recursion from most thread_lock consumers.  Return from
sched_add() without the thread_lock held.  This eliminates unnecessary
atomics and lock word loads as well as reducing the hold time for
scheduler locks.  This will eventually allow for lockless remote adds.

Discussed with: kib
Reviewed by:    jhb
Tested by:      pho
Differential Revision:  https://reviews.freebsd.org/D22626

HardenedBSD/hardenedbsd 1f84b28sys/powerpc/mpc85xx fsl_sata.c

powerpc/mpc85xx: Clean up Freescale SATA driver a little

* Remove unused ATA_IN/OUT macros, they just clutter up the file.
* Fix some RID management bits for the channel memory resource.

HardenedBSD/hardenedbsd 23ebecausr.bin/uniq uniq.c uniq.1

Support --all-repeats in uniq(1) for compatibility with gnu coreutils.

This adds a new -D/--all-repeats option to uniq(1), which outputs each copy
of any repeated lines (as opposed to a single copy of a repeated line). You
can specify a separator option to output a blank line before or after each
group of repeated lines. This adds compatibility with the GNU coreutils
version of uniq(1).

This change also re-groups the -c, -d, -D, -u options in the usage display
and man page to indicate that they are mutally exclusive of each other. This
matches the posix/opengroup definition of uniq(1) command line args. Note
that this change does NOT actually enforce the mutual exclusion in the code,
for now, it simply documents that the arguments should be considered
exclusive with each other.

Differential Revision:  https://reviews.freebsd.org/D22262

HardenedBSD/hardenedbsd 4f40920include stdlib.h, stand/libsa/zfs zfsimpl.c

Merge branch 'freebsd/current/master' into hardened/current/master

* freebsd/current/master:
  Revert r355760, r355759
  kbd: convert kbdd_* macros to inline functions
  Properly synchronize completion DMA buffers. Within command completion processing the 
callback function may access DMAed data buffer. Synchronize it before use, not after. This 
allows to use NVMe disk on non-DMA coherent arm64 system.
  loader: zfsimpl.c cstyle cleanup

HardenedBSD/hardenedbsd 5b3e917include stdlib.h, sys/compat/linuxkpi/common/include/linux compiler.h

Revert r355760, r355759

And remove the inline/deprecated attribute use entirely in stdlib.h, from
r355747.  The intent was to provide a buildable API transitionary period, but
clearly that was counter-productive.

Reported by:    delphij, imp, others

HardenedBSD/hardenedbsd 7dddb65sys/dev/kbd kbdreg.h

kbd: convert kbdd_* macros to inline functions

This reduces the noise when interested parties wish to de-Giant kbd; these
accesses to kbdsw will need to be properly locked.
DeltaFile
+140-47sys/dev/kbd/kbdreg.h
+140-471 files

HardenedBSD/hardenedbsd 61e28a2sys/dev/nvme nvme_qpair.c

Properly synchronize completion DMA buffers.
Within command completion processing the callback function may access
DMAed data buffer. Synchronize it before use, not after.
This allows to use NVMe disk on non-DMA coherent arm64 system.

MFC after:      3 weeks

HardenedBSD/hardenedbsd cfb1158stand/libsa/zfs zfsimpl.c

loader: zfsimpl.c cstyle cleanup

No functional changes intended.

MFC after:      1 week
DeltaFile
+139-117stand/libsa/zfs/zfsimpl.c
+139-1171 files

HardenedBSD/hardenedbsd b65c290sys/kern uipc_shm.c

Merge branch 'freebsd/current/master' into hardened/current/master

* freebsd/current/master:
  Fix a mistake in r355765.  We need to activate the page if it is not yet on a pagequeue.

HardenedBSD/hardenedbsd 86c2a38libexec/getty subr.c main.c, sbin/ipfw dummynet.c

Merge branch 'freebsd/12-stable/master' into hardened/12-stable/master

* freebsd/12-stable/master:
  MFC r355222, r355260:

HardenedBSD/hardenedbsd d9b4df0libexec/getty subr.c main.c, sbin/ipfw dummynet.c

MFC r355222, r355260:

r355222: Use strlcat().
r355260: Simplify code with strlcpy/strlcat.

HardenedBSD/hardenedbsd 671ed4esys/kern uipc_shm.c

Fix a mistake in r355765.  We need to activate the page if it is not yet
on a pagequeue.

Reported by:    pho

HardenedBSD/hardenedbsd fe1636esys/kern kern_exec.c, sys/vm vm_fault.c vm_page.c

Merge branch 'freebsd/current/master' into hardened/current/master

* freebsd/current/master:
  kbd: drop _KERNEL #ifdef in kbdreg.h
  Previously we did not support invalid pages in default objects.  This means that if 
fault fails to progress and needs to restart the loop it must free the page it is working 
on and allocate again on restart.  Resolve the few places that need to be modified to 
support this condition and simply deactivate the page.  Presently, we only permit this 
when fault restarts for busy contention.  This has an added benefit of removing some 
object trylocking in this case.
  Add a deferred free mechanism for freeing swap space that does not require an exclusive 
object lock.
  Slightly optimize locking in vm_map_copy_swap_entry().  Anonymous objects require the 
object lock to synchronize collapse.  Other swap objects such as tmpfs do not.
  Handle pagein clustering in vm_page_grab_valid() so that it can be used by 
exec_map_first_page().  This will also enable pagein clustering for other interested 
consumers (tmpfs, md, etc).
  cdefs: use more accurate GCC version for the deprecated attribute.
  <unistd.h>: remove redundant __BSD_VISIBLE

HardenedBSD/hardenedbsd 0de67ae. ObsoleteFiles.inc

Merge branch 'freebsd/12-stable/master' into hardened/12-stable/master

* freebsd/12-stable/master:
  MFC r355638: ObsoleteFiles.inc: chase libpcap update in r334277
DeltaFile
+2-0ObsoleteFiles.inc
+2-01 files

HardenedBSD/hardenedbsd 4041112sys/dev/kbd kbdreg.h

kbd: drop _KERNEL #ifdef in kbdreg.h

This #ifdef is misleading as there are actually no user-serviceable parts
inside and, as far as I can tell, there is no pollution leading from
userland to this header. Furthermore, it becomes a slight nuisance when
attempting to move things around in this header.

HardenedBSD/hardenedbsd 2644848. ObsoleteFiles.inc

MFC r355638:    ObsoleteFiles.inc: chase libpcap update in r334277

libpcap 1.9.0 (pre-release) update removed the export-defs.h header.

PR:            242559
Submitted by:   John Hein
DeltaFile
+2-0ObsoleteFiles.inc
+2-01 files

HardenedBSD/hardenedbsd 8ec40c1. ObsoleteFiles.inc

MFC r355638: ObsoleteFiles.inc: chase libpcap update in r334277

libpcap 1.9.0 (pre-release) update removed the export-defs.h header.

PR:            242559
Submitted by:   John Hein
DeltaFile
+2-0ObsoleteFiles.inc
+2-01 files

HardenedBSD/hardenedbsd 1cf4bc5sys/vm vm_fault.c vm_object.c

Previously we did not support invalid pages in default objects.  This means
that if fault fails to progress and needs to restart the loop it must free
the page it is working on and allocate again on restart.  Resolve the few
places that need to be modified to support this condition and simply
deactivate the page.  Presently, we only permit this when fault restarts
for busy contention.  This has an added benefit of removing some object
trylocking in this case.

While here consolidate some page cleanup logic into fault_page_free() and
fault_page_release() to reduce redundant code and automate some teardown.

Reviewed by:    kib
Differential Revision:  https://reviews.freebsd.org/D22653

HardenedBSD/hardenedbsd 011da14sys/dev/md md.c, sys/kern uipc_shm.c

Add a deferred free mechanism for freeing swap space that does not require
an exclusive object lock.

Previously swap space was freed on a best effort basis when a page that
had valid swap was dirtied, thus invalidating the swap copy.  This may be
done inconsistently and requires the object lock which is not always
convenient.

Instead, track when swap space is present.  The first dirty is responsible
for deleting space or setting PGA_SWAP_FREE which will trigger background
scans to free the swap space.

Simplify the locking in vm_fault_dirty() now that we can reliably identify
the first dirty.

Discussed with: alc, kib, markj
Differential Revision:  https://reviews.freebsd.org/D22654

HardenedBSD/hardenedbsd dae9c13sys/vm vm_map.c

Slightly optimize locking in vm_map_copy_swap_entry().  Anonymous objects
require the object lock to synchronize collapse.  Other swap objects such
as tmpfs do not.

Reported by:    mjg
Reviewed by:    kib, markj
Differential Revision:  https://reviews.freebsd.org/D22747
DeltaFile
+12-9sys/vm/vm_map.c
+12-91 files

HardenedBSD/hardenedbsd ed81eedsys/kern kern_exec.c, sys/vm vm_page.c

Handle pagein clustering in vm_page_grab_valid() so that it can be used by
exec_map_first_page().  This will also enable pagein clustering for other
interested consumers (tmpfs, md, etc).

Discussed with: alc
Approved by:    kib
Differential Revision:  https://reviews.freebsd.org/D22731

HardenedBSD/hardenedbsd 7c4d294sys/sys cdefs.h

cdefs: use more accurate GCC version for the deprecated attribute.

The message argument in the "deprecated" attribute was introduced in GCC 4.5 *.
Use the accurate version number for consistency, as done already with other
attributes.

* https://gcc.gnu.org/onlinedocs/gcc-4.5.0/gcc/Function-Attributes.html
DeltaFile
+1-1sys/sys/cdefs.h
+1-11 files

HardenedBSD/hardenedbsd da3c791include unistd.h

<unistd.h>: remove redundant __BSD_VISIBLE

This bit is already inside of a larger __BSD_VISIBLE block.

Reported by:    vangyzen
DeltaFile
+0-2include/unistd.h
+0-21 files

HardenedBSD/hardenedbsd 6734056include stdlib.h, sbin/mount_nfs mount_nfs.8

Merge branch 'freebsd/current/master' into hardened/current/master

* freebsd/current/master:
  linuxkpi: Drop incompatible __deprecated definition
  cdefs: Add __deprecated(message) function attribute macro
  Update the mount_nfs.8 man page to include NFSv4.2.
  Simplify the processing a leaf mask to find big-enough ranges of set bits, by storing 
and modifying the complement of the original leaf mask, and by avoiding some unnecessary 
intermediate variables in computing the shift amounts. The logic is similar to what has 
recently been committed to sys/sys/bitstring.h.

HardenedBSD/hardenedbsd d7e645fusr.bin/gcore elfcore.c

Merge branch 'freebsd/12-stable/master' into hardened/12-stable/master

* freebsd/12-stable/master:
  MFC r355506: gcore: Avoid using vm_map_entry_t.
DeltaFile
+27-20usr.bin/gcore/elfcore.c
+27-201 files

HardenedBSD/hardenedbsd 0399927sys/compat/linuxkpi/common/include/linux compiler.h

linuxkpi: Drop incompatible __deprecated definition

Probably all of these linuxkpi stubs should be '#ifndef' guarded, but maybe
that would prevent people from noticing when they are defined.

Introduced in r355759.  For some reason I only ran a buildworld and not a
kernel.  Mea culpa.

Reported by:    Mark Millard
X-MFC-with:     r355759

HardenedBSD/hardenedbsd cae007finclude stdlib.h, sys/sys cdefs.h

cdefs: Add __deprecated(message) function attribute macro

The legacy version of GCC4 currently in base does not support the
parameterized form of this function attribute, as recent introduced in
stdlib.h (r355747).

As we have done for other function attributes with similar compatibility
problems, add a version-compatibile definition in sys/cdefs.h.  Note that
Clang defines itself to be GCC 4, so one must check for __clang__ in
addition to __GNUC__ version.  On legacy GCC 4, the macro expands to just
the __deprecated__ attribute; on modern GCC or Clang, the macro expands to
the parameterized variant with the message.

Ignoring legacy or unsupported compilers, the macro is also beneficial in
that it is a bit more ergonomic than the full
__attribute__((__deprecated__())) boilerplate.

Reported by:    CI (but not tinderbox); imp and others
Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D22817

HardenedBSD/hardenedbsd bd6d416sbin/mount_nfs mount_nfs.8

Update the mount_nfs.8 man page to include NFSv4.2.

r355677 added NFSv4.2 support to the NFS client. This patch updates the
mount_nfs.8 man page to reflect that.
It also clarifies that the "nolockd" option does not apply to NFSv4 mounts.

This is a content change.

HardenedBSD/hardenedbsd e85455esys/kern subr_blist.c

Simplify the processing a leaf mask to find big-enough ranges of set
bits, by storing and modifying the complement of the original leaf
mask, and by avoiding some unnecessary intermediate variables in
computing the shift amounts. The logic is similar to what has recently
been committed to sys/sys/bitstring.h.

Compute better hint updates for the case when the cursor starts in
mid-leaf, and eliminates some otherwise viable solutions. Assume the
worst case, that all the eliminated offsets could have been solutions,
and you can still compute a better hint than we use now.

Eliminate some unnecessary conditional control flow.

Approved by: alc
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D22666
DeltaFile
+35-66sys/kern/subr_blist.c
+35-661 files

HardenedBSD/hardenedbsd b325fbcusr.bin/gcore elfcore.c

MFC r355506:
gcore: Avoid using vm_map_entry_t.
DeltaFile
+27-20usr.bin/gcore/elfcore.c
+27-201 files

HardenedBSD/hardenedbsd 71cc7b9usr.bin/nc Makefile

HBSD: Resolve merge conflict

Signed-off-by:  Shawn Webb <shawn.webb at hardenedbsd.org>

HardenedBSD/hardenedbsd f6d17d1contrib/netcat netcat.c, sys/amd64/linux linux_systrace_args.c

Merge remote-tracking branch 'origin/freebsd/current/master' into hardened/current/master

Conflicts:
        usr.bin/nc/Makefile (unresolved)