Add S3 access keys
An S3 access key is the SigV4 credential pair a client signs requests
to the TrueNAS S3 service with. It gets its own table and the new
s3.accesskey namespace rather than a place in the API key table, so
nothing in the TrueNAS API authentication path ever reads it and the
security sensitive api_key surface stays untouched.
A key belongs to a local or directory services account through the
same user_identifier linkage API keys use, resolved to a username at
query time, and needs no privilege roles because the S3 service runs
its requests as the account. The access key id and the secret are
generated unless supplied, both in the character sets the S3 service
config reader carries verbatim. The secret is stored recoverable
because SigV4 derives signing keys from it, and it stays readable to
administrators holding the new SHARING_S3_WRITE role while being
redacted for everyone else. A computed status folds the enabled flag,
expiry, a deleted account and a secret lost to a config restore
without the secret seed into one value, so the credentials file will
[8 lines not shown]
arc: harness buf_hdr's anon state invariant checks
There are several places where hdr is checked for arc_anon state
without the hash_lock, for example at arc_release() or at
arc_buf_destroy(), and it's supposed that the hdr must not be
in the hash table in this state and not having IO_IN_PROGRESS.
However, those invariants are not asserted in release builds.
But even if they would, there is one place in the current code
where they would pass without noticing the problem.
This place is at arc_write_done() where we first insert hdr
into the hash table, then clear IO_IN_PROGRESS flag, then drop
the reference and only then move its state from anon to mru:
7048 exists = buf_hash_insert(hdr, &hash_lock); /* still anon */
...
7081 arc_hdr_clear_flags(hdr, ARC_FLAG_IO_IN_PROGRESS);
7082 VERIFY3S(remove_reference(hdr, hdr), >, 0); /* ref: 2 -> 1 */
7084 if (exists == NULL && hdr->b_l1hdr.b_state == arc_anon)
[34 lines not shown]
Redact uploaded license from the audit trail
This commit fixes an issue where `truenas.license.upload` recorded its license argument verbatim in the audit trail, because the field was never declared `Secret[...]`. Uploading needs FULL_ADMIN but audit records come back with SYSTEM_AUDIT_READ, so a read-only admin could pull out the blob a full admin installed, and the same record went to remote syslog and into debug bundles.
Marking the field `Secret[LongNonEmptyString]` means `check_annotations` stops accepting the old `license_: str` signature, and the `str(license_)` that went with it turned out to be wrong already: it stringified the `LongStringWrapper` and wrote `LongStringWrapper(<pem>)` into the license file. The daemon's PEM scan skips the leading garbage so nothing ever failed, but the on-disk copy never matched what was uploaded, which defeats the TNC heartbeat's raw_license comparison and makes every HA send_license add another wrapper layer. Unwrapping both boxes writes the PEM back byte for byte.
zstd: declare __asan_*_memory_region() for user space builds
ZSTD_ASAN_POISON() and ZSTD_ASAN_UNPOISON() call
__asan_poison_memory_region() and __asan_unpoison_memory_region()
directly, but nothing declares them in the user space build. The kernel
block just above declares both -- and stubs them out, since KASAN does
not provide them -- while the user space block was left with only the
macros.
That builds where an implicit function declaration is a warning, since
the real symbols come from libasan at link time, but fails on compilers
that make it an error, as GCC 14 and newer do by default:
zfs_zstd.c:278:34: error: implicit declaration of function
'__asan_poison_memory_region' [-Wimplicit-function-declaration]
Declare both in the user space block, mirroring the kernel block.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19017
[AArch64] Use NoSchedPred for SME instructions in C1 scheduling models. (#220553)
We can have the scheduling model enabled without SME using -mtune, which
means that no scheduling information was present for any instructions
that execute in either SME or the core. AFAICT the predicate should be
NoSchedPred, as any instructions should be using the non-streaming
scheduling info when not in a SME function.
Fixes #220070
Fixes #220067
[orc-rt] Drop the _C_ prefix from ORC_RT_C_FORMAT_PRINTF (#220592)
The macro is not C-specific. Rename it to ORC_RT_FORMAT_PRINTF and
update its two uses in Logging.h.
[flang][cuda] Delay descriptor alloc when addressed reused on host/device (#220534)
CSE can share one fir.coordinate_of between the host-association capture
store and a later fir.load. Treating that coordinate_of as a real use
made cuf-alloc-delay think the movable group depended on an operand at
the sink point, so the device descriptor stayed at function entry and
cudaMallocManaged ran before cudaSetDevice.
Count only users of the slot address that actually read it. Stores that
populate the tuple still sink with the allocation group.
Add S3 access keys
An S3 access key is the SigV4 credential pair a client signs requests
to the TrueNAS S3 service with. It gets its own table and the new
s3.accesskey namespace rather than a place in the API key table, so
nothing in the TrueNAS API authentication path ever reads it and the
security sensitive api_key surface stays untouched.
A key belongs to a local or directory services account through the
same user_identifier linkage API keys use, resolved to a username at
query time, and needs no privilege roles because the S3 service runs
its requests as the account. The access key id and the secret are
generated unless supplied, both in the character sets the S3 service
config reader carries verbatim. The secret is stored recoverable
because SigV4 derives signing keys from it, and it stays readable to
administrators holding the new SHARING_S3_WRITE role while being
redacted for everyone else. A computed status folds the enabled flag,
expiry, a deleted account and a secret lost to a config restore
without the secret seed into one value, so the credentials file will
[6 lines not shown]
[VPlan] Allow non-live-in IV offsets when simplifying latch cond (NFC). (#220734)
simplifyBranchConditionForVFAndUF matches the canonical IV increment
plus an offset, which epilogue vectorization adds to resume the
canonical IV at the vector trip count of the main vector loop. Require
the offset to be defined outside the vector loop region instead of
requiring it to be a live-in; that is what makes it available in the
preheader..
This is NFC today, but prepares for modeling the full epilogue skeleton
in VPlan, which requires adding phi nodes in the preheader before
execute.
x11/florence: Update to 0.7.2 and adopt
Switch from the stalled SourceForge 0.6.3 distfile to the FrauBSD
continuation on GitHub and take maintainership from ports@.
Drop PORTREVISION, fetch via USE_GITHUB.
Default-on FREEBSD_ORB option replaces the Super key glyph with the
FreeBSD orb (ports-only branding).
Reviewed by: jrm
Differential Revision: https://reviews.freebsd.org/D58428
unit: add nvpair test suite
Exercises add and lookup for all types, iteration, removal, replacement,
merge & copy, pack & unpack with both encodings, formatting and
infallibile wrappers.
Sponsored-by: TrueNAS
Reviewed-by: Chris Longros <chris.longros at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19006
unit: add null/notnull helpers
Sponsored-by: TrueNAS
Reviewed-by: Chris Longros <chris.longros at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19006
[LoopVersioning] Add missing verify-analysis-invalidation=false to test. (#220735)
Add -verify-analysis-invalidation=false to test added in
https://github.com/llvm/llvm-project/pull/220537 to fix expensive check
failures due to extra verification passes.
[lldb] Pass Wasm runtime-args before the port argument (#220700)
A runtime that dispatches on a leading subcommand, such as WasmKit's
`wasmkit run`, could not be driven directly: runtime-args landed after
the port argument, so the subcommand did too and the runtime rejected
it. Naming the subcommand required a wrapper script. Move runtime-args
ahead of the port argument so the setting can carry it.
Extract the command line assembly into PlatformWasm::MakeRuntimeCommand
so the ordering is covered by unit tests, and clarify that port-arg has
to carry its value in the same argument.
unit: make unit_eq() etc type-aware
Previously, unit_eq() and friends were naively routed to
munit_assert_uint64(), regardless of the actual types involved. This
meant that on failure, the error message would report eg a small
negative number as a huge unsigned one, and also plain doesn't work for
types that don't have an implicit conversion to uint64_t, like double.
Here we add a type-aware replacement that generates comparison and
failure reporting functions for the wanted types, then uses _Generic to
dispatch to the correct one.
Sponsored-by: TrueNAS
Reviewed-by: Chris Longros <chris.longros at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19006
[llvm] Use ValueMap for ValueToGUIDMap (#220682)
ValueToGUIDMap currently uses a DenseMap which does not properly track
the deletion of Values, leaving dangling pointers in the map.
This change fixes this by using ValueMap. FollowRAUW is set to false to
match the current behavior of DenseMap.
This bug was discovered by a sanity test for deterministic compilation
where, depending on the allocator state, a newly allocated Value could
re-use the address of a previously deleted one, incorrectly inheriting
the GUID.
[VPlan][Predicator] Preserve some uniform control flow
Implements "Partial Control-Flow Linearization" by Simon Moll and
Sebastian Hack.
That should allow implementation of an alternative to
https://github.com/llvm/llvm-project/pull/141900 based on this
functionality (see BOSCC in the paper).
Inherit the project ID for every object type
zfs_mknode() only assigned a project ID to regular files and
directories, so a symlink, device node, FIFO or socket created inside a
directory with ZFS_PROJINHERIT set was left at ZFS_DEFAULT_PROJID. The
cross-project checks in zfs_rename() and zfs_link() compare the object's
project ID against the directory's, so such an object is treated as
foreign to the very directory holding it, and cannot be renamed or
linked there at all -- "ln -sfn", which creates the new symlink under a
temporary name and renames it into place, fails with EXDEV.
ext4 and XFS store a project ID on every inode type and so do not have
this problem. Do the same, and quota-check new symlinks and rename
whiteouts against the inherited ID rather than the default one, so their
space is accounted to the project that owns them.
This changes accounting for newly created objects: symlinks, device
nodes and FIFOs now consume the project's quota where they previously
consumed none, so on a dataset already at its project quota, creating
[12 lines not shown]
Allow renames within a single directory under project inheritance
A rename that keeps the object in the directory it already lives in
cannot move it between projects, so refusing it with EXDEV is never
right. Objects created before the previous commit carry no project ID of
their own, which makes this reachable on existing pools: a symlink in a
project directory cannot be renamed even to another name beside itself,
and "ln -sfn" over an existing path fails.
zfs_link() keeps its unconditional check. It has no source directory to
compare against -- a hard link names an object that may live anywhere --
so there is no equivalent "the object is already here" case to exempt.
Linking a pre-existing symlink, device node or FIFO into the project
directory that already holds it therefore still fails with EXDEV, until
the object is given a project ID of its own.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matt Turner <mattst88 at gmail.com>
Closes #18932
ZTS: check project ID inheritance and renames within a project directory
Covers both of the preceding changes. For the inheritance: a new symlink
and a new FIFO each add an object to the containing directory's project,
which only happens once non-regular files inherit a project ID, and a
symlink can then be renamed into a different directory carrying the same
project ID, which rename(2) permits only when the two project IDs match.
For the rename exemption: a regular file and a symlink are created
before the directory is tagged, so they carry no project ID of their
own, as every symlink on an existing pool does. Renaming each of them
within that directory afterwards fails with EXDEV unless the
cross-project check exempts renames whose source and target directories
are the same. The same renames are then repeated for objects created
after the tagging, along with replacing a symlink with "ln -sfn", and
again in an inheriting subdirectory.
A rename that does cross into a different project is still refused.
[7 lines not shown]
[libclc] Fix remainder calculation in clc_remquo for subnormals (#217925)
The remainder t was previously computed using:
__CLC_GENTYPE t = __clc_mad(y, -__CLC_CONVERT_GENTYPE(qsgn), x);
Multiplying y by -qsgn (+-1.0) introduces an unnecessary intermediate
multiplication step. On platforms or execution modes where subnormals
are flushed to zero computing `y * -qsgn` can prematurely flush a
subnormal `y` to zero, resulting in `0.0 + x = x` instead of computing
the subtraction `x - y` (or `x + y`).
Replace `__clc_mad` with a direct addition/subtraction based on the sign
of the quotient:
__CLC_GENTYPE t = qsgn > 0 ? (x - y) : (x + y);
This avoids multiplication by +-1.0, eliminates unwanted subnormal
flushing on intermediate products in FTZ modes, and computes the exact
remainder.
