mfc-candidates: Convert to Lua
d51c59002367 added a Lua script to process the lists of candidate and
completed MFC commits to address sorting issues in the original shell
implementation.
Instead of having a mix of shell and Lua, just implement the entire
tool in Lua. This is more maintainable and gives a reasonable
improvement in performace.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47416
Change the SOCKBUF_LOCK calls to use the more refined SOCK_XXXBUF_LOCK/UNLOCK.
The socket buffer locking used to be standard on SOCKBUF_LOCK/UNLOCK. But we are now
moving to a more elegant SOCK_SENDBUF_LOCK/UNLOCK and SOCK_RECVBUF_LOCK/UNLOCK.
Lets get BBR and Rack to use these updated macros.
Reviewed by:glebius, tuexen, rscheff
Differential Revision:https://reviews.freebsd.org/D47542
aio: remove write-only jobid & kernelinfo
The jobid (which was stored in kernelinfo) was used to look up
jobs until 1ce9182407f6, where it became essentially write only.
Remove it to simplify the code and pave the way for future work
to make aio scale better.
Note this has been slated for removal "soon" for 18 years.
Suggested by: jhb
Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D47583
cred: crsetgroups(): Throw away old groups before crextend()
Now that crextend() asserts that groups are not set (rightfully so,
since it may change the backing storage without copying the content of
the old one), have crsetgroups() throw away the old groups before
calling it, as it installs an entirely new set anyway.
This allows to reuse unshared credentials by resetting their groups set,
as NFS exports actually do.
Reported by: cy
Tested by: cy
Fixes: ea26c0e79752 ("cred: crextend(): Harden, simplify")
Pointy hat to: olce
While here, as I forgot these credits in commit 5169d4307eb9 ("nfs: Fallback to
GID_NOGROUP on no groups"):
Tested by: cy, David Wolfskill (panics caused by mountd(8))
Tested by: kib (MINIMAL/custom kernel compile breakup)
[4 lines not shown]
cred: groupmember(): Remove tolerance for empty 'cr_groups'
This tolerance had been introduced in commit caa309c8811d ("nfsd: Fix
handling of credentials with cr_ngroups == 0", on 2024/10/21).
Now that NFS has been modified in the previous commit (cfbe7a62dc62,
"nfs, rpc: Ensure kernel credentials have at least one group") to rule
out credentials with empty 'cr_groups' (and thus, no 'cr_gid'), remove
it back.
This basically reverts the above-mentioned commit except for the not
directly related modifications it contains, which are still valid.
Discussed with: rmacklem (by mail)
Approved by: markj (mentor)
MFC after: 3 days
(cherry picked from commit 3f313682e030f1c99d9b223093e84d0a53a109af)
Approved by: markj (mentor)
nfs: Fallback to GID_NOGROUP on no groups
We cannot unconditionally access nfsd's VNET variables in
'sys/kern/vfs_export.c' nor 'sys/fs/nfsserver/nfs_nfsdsubs.c', as they
may not have been compiled in depending on build options.
So, forget about the extra mile of using the configured default group
and use the hardcoded GID_NOGROUP (which differs only on systems running
nfsuserd(8) and with a non-default GID for their "nogroup" group).
Reported by: rpokala, bapt (MINIMAL compile breakup)
Reported by: cy, David Wolfskill (panics caused by mountd(8))
Approved by: markj (mentor)
Fixes: cfbe7a62dc62 ("nfs, rpc: Ensure kernel credentials have at least one group")
(cherry picked from commit 5169d4307eb9c8b7bb0bd46d600012bcc12cbdae)
cred: groupmember() and co.: Sanity check cred's groups (INVARIANTS)
Leverage the normalization check functions introduced in the previous
commit in all public-facing groups search functions to catch programming
errors early.
Approved by: markj (mentor)
MFC after: 3 days
(cherry picked from commit 634675067867090e538b08e62ff9b14d3ffae5a3)
Approved by: markj (mentor)
cred: crextend(): Harden, simplify
Harden by adding more assertions, and a plain panic in case of an
unrepresentable size for the groups array (this can never happen after
the change of the 'kern.ngroups' computation to impose some not too high
maximum value a few commits ago). Fix an impact in kern_setgroups().
Simplify by removing the iterative process whose purpose is actually to
determine the closest power of two that is greater than the wanted
number of bytes. Using the proper target quantity (number of bytes)
incidentally helps with eliminating divisions (and the reliance on
sizeof(gid_t) being a power of two).
Reviewed by: mhorne (older version)
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46915
(cherry picked from commit ea26c0e797525634dd25dede233ff2ded053cf2d)
[2 lines not shown]
cred: New crsetgroups_fallback()
Similar to crsetgroups(), but allows an empty group array in input,
treating it like a one-element array containing the passed fallback
group.
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46917
(cherry picked from commit d4e0d4d92e010b74404bddc87c1a1e8dbcaa25dd)
Approved by: markj (mentor)
nfs, rpc: Ensure kernel credentials have at least one group
This fixes several bugs where some 'struct ucred' in the kernel,
constructed from user input (via nmount(2)) or obtained from other
servers (e.g., gssd(8)), could have an unfilled 'cr_groups' field and
whose 'cr_groups[0]' (or 'cr_gid', which is an alias) was later
accessed, causing an uninitialized access giving random access rights.
Use crsetgroups_fallback() to enforce a fallback group when possible.
For NFS, the chosen fallback group is that of the NFS server in the
current VNET (NFSD_VNET(nfsrv_defaultgid)).
There does not seem to be any sensible fallback available in rpc code
(sys/rpc/svc_auth.c, svc_getcred()) on AUTH_UNIX (TLS or not), so just
fail credential retrieval there. Stock NSS sources, rpc.tlsservd(8) or
rpc.tlsclntd(8) provide non-empty group lists, so will not be impacted.
Discussed with: rmacklem (by mail)
Approved by: markj (mentor)
[6 lines not shown]
cred: kern_setgroups(): Internally use int as number of groups' type
sys_setgroups() (and sys_getgroups()) was changed in commit "kern: fail
getgroup and setgroup with negative int" (4bc2174a1b48) to take the
number of groups as an 'int' (for sys_getgroups(), POSIX mandates this
change; for sys_setgroups(), which it does not standardize, it's
arguably for consistency).
All our internal APIs related to groups on 'struct ucred', as well as
related members on the latter, treat that number as an 'int' as well
(and not a 'u_int').
Consequently, to avoid surprises, change kern_setgroups() to behave the
same, and fix audit_arg_groupset() accordingly. With that change,
everything is handled with signed integers internally.
Update sanity checks accordingly.
Reviewed by: mhorne
[7 lines not shown]
cred: Separate constant for the number of inlined groups
CRED_SMALLGROUPS_NB now holds the number of inlined groups in field
'cr_smallgroups'. XU_NGROUPS stays the number of groups allowed in
'struct xucred'. The first is an implementation detail, while the
second is part of a public interface. All mentions of XU_NGROUPS in the
tree have been reviewed and only those concerning the implementation
detail have been changed to use CRED_SMALLGROUPS_NB (they were all in
'kern_prot.c').
No functional change, as CRED_SMALLGROUPS_NB is set to 16, the same
value as XU_NGROUPS.
Reviewed by: mhorne (slightly different version)
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46911
(cherry picked from commit 664b9fcb1c051c17ba11d1e5e8a1db9938d76bd5)
[2 lines not shown]
cred: 'kern.ngroups' tunable: Limit it to avoid internal overflows
As the comment introduced with the tunable said (but the code didn't
do), make sure that 'ngroups_max' can't be INT_MAX, as this would cause
overflow in the usual 'ngroups_max + 1' computations (as we store the
effective GID and supplementary groups' IDs in the same array, and
'ngroups_max' only applies to supplementary groups).
Further, we limit the maximum number of groups somewhat arbitrarily to
~17M so as to avoid overflow when computing the size in bytes of the
groups set's backing array and to avoid obvious configuration errors.
We really don't think that more than ~17M groups will ever be needed (if
I'm proven wrong one day, please drop me a note about your use case).
While here, document more precisely why NGROUPS_MAX needs to be the
minimum value for 'ngroups_max'.
Reviewed by: mhorne (older version)
Approved by: markj (mentor)
[6 lines not shown]
cred: crsetgroups(): Improve and factor out groups normalization
The groups array has been sorted (not including the first element, which
is always the effective GID) to enable performing a binary search for
determining if some group is part of the supplementary groups set.
Factor out this sorting operation into an internal normalization
function (groups_normalize()), adding to it the removal of duplicates
after the sort.
Separating groups normalization code allows to perform it in advance,
and in particular before calling MAC hooks which need the groups array
to be sorted to perform better. This also enables sorting input arrays
ahead of acquiring the process lock (which is not necessary for this
operation).
kern_setgroups() has been changed accordingly, so MAC modules
implementing the mac_cred_check_setgroups() hook now can assume
a normalized groups array (and also that it has at least one element, as
[32 lines not shown]
cred: group_is_supplementary(): Use bsearch()
This makes that function use a more efficient version of binary search
instead, and removes one more hand-rolled binary search code from the
tree (and the kernel binary).
Reviewed by: mhorne, emaste
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46907
(cherry picked from commit 372605478c6fe2c628f25428af201f866d7eb015)
Approved by: markj (mentor)
cred: supplementary_group_member() => group_is_supplementary()
The new name seems more immediately descriptive.
While here, constify its signature ('struct ucred' isn't modified).
While here, replace "supplemental" => "supplementary" in some comments.
No functional change (intended).
Reviewed by: mhorne, emaste
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46906
(cherry picked from commit 2e031fd0719b567e66198bc76c8a1c6c9b7f2920)
Approved by: markj (mentor)
cred: crsetgroups(): Throw away old groups before crextend()
Now that crextend() asserts that groups are not set (rightfully so,
since it may change the backing storage without copying the content of
the old one), have crsetgroups() throw away the old groups before
calling it, as it installs an entirely new set anyway.
This allows to reuse unshared credentials by resetting their groups set,
as NFS exports actually do.
Reported by: cy
Tested by: cy
Fixes: ea26c0e79752 ("cred: crextend(): Harden, simplify")
Pointy hat to: olce
While here, as I forgot these credits in commit 5169d4307eb9 ("nfs:
Fallback to GID_NOGROUP on no groups"):
Tested by: cy, David Wolfskill (panics caused by mountd(8))
Tested by: kib (MINIMAL/custom kernel compile breakup)
[4 lines not shown]
cred: groupmember(): Remove tolerance for empty 'cr_groups'
This tolerance had been introduced in commit caa309c8811d ("nfsd: Fix
handling of credentials with cr_ngroups == 0", on 2024/10/21).
Now that NFS has been modified in the previous commit (cfbe7a62dc62,
"nfs, rpc: Ensure kernel credentials have at least one group") to rule
out credentials with empty 'cr_groups' (and thus, no 'cr_gid'), remove
it back.
This basically reverts the above-mentioned commit except for the not
directly related modifications it contains, which are still valid.
Discussed with: rmacklem (by mail)
Approved by: markj (mentor)
MFC after: 3 days
(cherry picked from commit 3f313682e030f1c99d9b223093e84d0a53a109af)
Approved by: markj (mentor)
nfs, rpc: Ensure kernel credentials have at least one group
This fixes several bugs where some 'struct ucred' in the kernel,
constructed from user input (via nmount(2)) or obtained from other
servers (e.g., gssd(8)), could have an unfilled 'cr_groups' field and
whose 'cr_groups[0]' (or 'cr_gid', which is an alias) was later
accessed, causing an uninitialized access giving random access rights.
Use crsetgroups_fallback() to enforce a fallback group when possible.
For NFS, the chosen fallback group is that of the NFS server in the
current VNET (NFSD_VNET(nfsrv_defaultgid)).
There does not seem to be any sensible fallback available in rpc code
(sys/rpc/svc_auth.c, svc_getcred()) on AUTH_UNIX (TLS or not), so just
fail credential retrieval there. Stock NSS sources, rpc.tlsservd(8) or
rpc.tlsclntd(8) provide non-empty group lists, so will not be impacted.
Discussed with: rmacklem (by mail)
Approved by: markj (mentor)
[6 lines not shown]
nfs: Fallback to GID_NOGROUP on no groups
We cannot unconditionally access nfsd's VNET variables in
'sys/kern/vfs_export.c' nor 'sys/fs/nfsserver/nfs_nfsdsubs.c', as they
may not have been compiled in depending on build options.
So, forget about the extra mile of using the configured default group
and use the hardcoded GID_NOGROUP (which differs only on systems running
nfsuserd(8) and with a non-default GID for their "nogroup" group).
Reported by: rpokala, bapt (MINIMAL compile breakup)
Reported by: cy, David Wolfskill (panics caused by mountd(8))
Approved by: markj (mentor)
Fixes: cfbe7a62dc62 ("nfs, rpc: Ensure kernel credentials have at least one group")
(cherry picked from commit 5169d4307eb9c8b7bb0bd46d600012bcc12cbdae)
Approved by: markj (mentor)
cred: crsetgroups(): Improve and factor out groups normalization
The groups array has been sorted (not including the first element, which
is always the effective GID) to enable performing a binary search for
determining if some group is part of the supplementary groups set.
Factor out this sorting operation into an internal normalization
function (groups_normalize()), adding to it the removal of duplicates
after the sort.
Separating groups normalization code allows to perform it in advance,
and in particular before calling MAC hooks which need the groups array
to be sorted to perform better. This also enables sorting input arrays
ahead of acquiring the process lock (which is not necessary for this
operation).
kern_setgroups() has been changed accordingly, so MAC modules
implementing the mac_cred_check_setgroups() hook now can assume
a normalized groups array (and also that it has at least one element, as
[32 lines not shown]
cred: New crsetgroups_fallback()
Similar to crsetgroups(), but allows an empty group array in input,
treating it like a one-element array containing the passed fallback
group.
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46917
(cherry picked from commit d4e0d4d92e010b74404bddc87c1a1e8dbcaa25dd)
Approved by: markj (mentor)
cred: crextend(): Harden, simplify
Harden by adding more assertions, and a plain panic in case of an
unrepresentable size for the groups array (this can never happen after
the change of the 'kern.ngroups' computation to impose some not too high
maximum value a few commits ago). Fix an impact in kern_setgroups().
Simplify by removing the iterative process whose purpose is actually to
determine the closest power of two that is greater than the wanted
number of bytes. Using the proper target quantity (number of bytes)
incidentally helps with eliminating divisions (and the reliance on
sizeof(gid_t) being a power of two).
Reviewed by: mhorne (older version)
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46915
(cherry picked from commit ea26c0e797525634dd25dede233ff2ded053cf2d)
[2 lines not shown]
cred: groupmember() and co.: Sanity check cred's groups (INVARIANTS)
Leverage the normalization check functions introduced in the previous
commit in all public-facing groups search functions to catch programming
errors early.
Approved by: markj (mentor)
MFC after: 3 days
(cherry picked from commit 634675067867090e538b08e62ff9b14d3ffae5a3)
Approved by: markj (mentor)
cred: 'kern.ngroups' tunable: Limit it to avoid internal overflows
As the comment introduced with the tunable said (but the code didn't
do), make sure that 'ngroups_max' can't be INT_MAX, as this would cause
overflow in the usual 'ngroups_max + 1' computations (as we store the
effective GID and supplementary groups' IDs in the same array, and
'ngroups_max' only applies to supplementary groups).
Further, we limit the maximum number of groups somewhat arbitrarily to
~17M so as to avoid overflow when computing the size in bytes of the
groups set's backing array and to avoid obvious configuration errors.
We really don't think that more than ~17M groups will ever be needed (if
I'm proven wrong one day, please drop me a note about your use case).
While here, document more precisely why NGROUPS_MAX needs to be the
minimum value for 'ngroups_max'.
Reviewed by: mhorne (older version)
Approved by: markj (mentor)
[6 lines not shown]
cred: kern_setgroups(): Internally use int as number of groups' type
sys_setgroups() (and sys_getgroups()) was changed in commit "kern: fail
getgroup and setgroup with negative int" (4bc2174a1b48) to take the
number of groups as an 'int' (for sys_getgroups(), POSIX mandates this
change; for sys_setgroups(), which it does not standardize, it's
arguably for consistency).
All our internal APIs related to groups on 'struct ucred', as well as
related members on the latter, treat that number as an 'int' as well
(and not a 'u_int').
Consequently, to avoid surprises, change kern_setgroups() to behave the
same, and fix audit_arg_groupset() accordingly. With that change,
everything is handled with signed integers internally.
Update sanity checks accordingly.
Reviewed by: mhorne
[7 lines not shown]
cred: Separate constant for the number of inlined groups
CRED_SMALLGROUPS_NB now holds the number of inlined groups in field
'cr_smallgroups'. XU_NGROUPS stays the number of groups allowed in
'struct xucred'. The first is an implementation detail, while the
second is part of a public interface. All mentions of XU_NGROUPS in the
tree have been reviewed and only those concerning the implementation
detail have been changed to use CRED_SMALLGROUPS_NB (they were all in
'kern_prot.c').
No functional change, as CRED_SMALLGROUPS_NB is set to 16, the same
value as XU_NGROUPS.
Reviewed by: mhorne (slightly different version)
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46911
(cherry picked from commit 664b9fcb1c051c17ba11d1e5e8a1db9938d76bd5)
[2 lines not shown]
