zio_crypt_os: implement platform interface for FreeBSD
This uses our internal "crypto_os" shim as the backend to the FreeBSD
kernel crypto. There's no particular reason this separate shim needs to
exist anymore, but we keep it for now to keep this interface small.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18884
zio_crypt_os: implement platform interface for ICP
This is now a generic implementation for any platform that wants to use
the ICP for its crypto. It exists in the "common" module source, linked
and wired for Linux and libzpool.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18884
zio_crypt: establish platform interface; rework common code to use it
This lifts out and generalises the common parts of linux/zio_crypt and
freebsd/zio_crypt into a common version, with a
platform/backend-specific API for the not-common parts.
The common parts here involve key management, data assembly for
encrypt/decrypt, on-disk formats and so on - the "logic" side of the
equation, which are subtle and definitely shouldn't be duplicated. The
platform-specific parts meanwhile are mostly just glue to get in and out
of the platform-provided cryptographic suite.
To be clear - this is not a particular _good_ API, but further
improvement requires changes in the logic code. Establishing a clear
boundary will allow that change to be worked on more safely.
In the header, we add platform-specific types for the two places where
incompatible concepts leaked through from the implementations:
[35 lines not shown]
zio_crypt: add platform-specific headers
As we pull platform-specific things out of zio_crypt, we'll need
somewhere to move things to.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18884
zio_crypt: single field for mechname
On FreeBSD, it's only used in debug output anyway; there's definitely
no need for it to not just be a normal string in both.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18884
linux/uio: don't require wrapped data to be const
This makes it harder to work with zfs_uio_t internally ourselves, for no
apparently good reason. If a caller has a const iovec/bvec that they
want to wrap in a uio, its up to them to either cast away the const or
copy the data as appropriate. As it is, there does not appear to be any
places within OpenZFS that require such action.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18884
uio: add zfs_uio_iov() accessor for uio_iov
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18884
NAS-143743 / 27.0.0-BETA.1 / Add truenas.entitlements.facts and deprecate system.product_type (#19704)
This commit adds changes to expose the hardware and license facts that
entitlement decisions are computed from through a new
truenas.entitlements.facts method, and marks system.product_type
removed_in v27 now that it has a proper successor. Hardware is reported
as TRUENAS or COMMUNITY off is_appliance, so a Mini reads as COMMUNITY
exactly like product_type reported it, and license_type is left as a
plain string so a newer license type doesn't fail validation on an older
system.
The models are mirrored into both v26 and v27 since this gets
backported. Usage reporting keeps emitting the old
ENTERPRISE/COMMUNITY_EDITION platform string after moving off
product_type, otherwise every existing usage report would stop lining up
with the new ones.
[SPIR-V] Add -spirv-nonsemantic-debug-info-version to select the NSDI set
The backend hardcoded NonSemantic.Shader.DebugInfo.100 as its debug-info
ext-inst set. Add a flag selecting .100 (the default) or .200.
[OpenMP] Remove the LocationDescription constructor that dropped the location. (#221949)
LocationDescription had an implicit conversion from a bare insertion
point, so `ompBuilder->emitSomething(someInsertPoint, ...)` compiled
happily and silently produced an empty debug location. Because
updateToLocation() installs the location unconditionally, this was worse
than a missing assignment: it cleared whatever the builder was carrying,
and the emitted runtime call ended up with no !dbg. On the device those
calls are inlinable, so the verifier rejects them once the runtime
carries debug info -- which is how this kept turning up as bug reports
rather than as anything visible at the callsite.
With the callers in OMPIRBuilder, OpenMPOpt, the MLIR translation and
clang all converted, the constructor can go, and the compiler will now
refuse the shape that caused the problem. Callers have to say which
location they mean: pass the IRBuilder to take its current one, or spell
out the insertion point and location as a pair.
Co-authored-by: Cursor <cursoragent at cursor.com>
[AMDGPU] Update no-modifier operand tests for the dropped align diagnostic
The no-modifier reg-or-inline operands routed through the HwMode
predicate now report a misaligned tuple as a plain invalid operand,
matching the diagnostic dropped earlier in the stack.
[AMDGPU] Route no-modifier reg-or-inline AsmParser operands through HwMode predicate
Convert the reg-or-inline operands with no modifiers (MFMA VGPR/AGPR
sources, VCSrc, v_pk_mov_b32, VOP scalar f64) from the fixed-class
isRegOrInlineNoMods to the HwMode-aware isRegOrInlineNoModsByHwMode, so an
odd-aligned tuple is rejected at the offending operand column instead of by
the validateVGPRAlign catch-all.
Co-Authored-By: Claude <noreply at anthropic.com>
[mlir][NFC] Prune unused copyArrayRefInto template (#222365)
`copyArrayRefInto` has internal linkage, so it is only reachable from
MLIRContext.cpp, and it has no callers there. It is the only mention of
the name in the MLIR tree.
An unused internal-linkage template fires -Wunused-template. That
warning is not in -Wall on main today, having been reverted by
8710728e0418, but it did ship enabled in clang 23.1.0, so building MLIR
with that released compiler under -Werror fails here:
mlir/lib/IR/MLIRContext.cpp:379:20: error: unused function template
'copyArrayRefInto' [-Werror,-Wunused-template]
This is the same deletion as one hunk of #221480, which cleans up the
same warning across 21 MLIR files and is still open. Landing that PR is
the better outcome; this is split out only because the file blocks a
release/23.x build today.
[2 lines not shown]
[InferAddressSpaces] Safely delete duplicate dead instructions (#222115)
Track deletion candidates with WeakTrackingVH so recursive deletion
cannot leave dangling pointers in later duplicate or overlapping
worklist entries.
[AMDGPU] Make custom AsmParser matchers alignment aware (#221988)
Currently custom matchers accept registers belonging to unaligned classes, leaving the alignment check and its diagnostic to validateVGPRAlign. The problem is that validateVGPRAlign does not account for the operand register class, which may have a different alignment requirement on mixed-alignment targets.
The fix is to add a custom matcher that resolves the operand's `_AlignTarget` class via HwMode and uses that resolved class to accept a register. The drawback is that this changes the diagnostic from the more specific "error: invalid register class: vgpr tuples must be 64 bit aligned" to the generic "error: invalid operand for instruction" — but keeping the specific one doesn't seem worth the effort.
This commit converts the input-mods and DP-ALU DPP matchers. The no-modifier reg-or-inline operands come next. validateVGPRAlign stays as a catch-all until every operand is converted.
Co-Authored-By: Claude [noreply at anthropic.com](mailto:noreply at anthropic.com)
Fix metaslab count assertion in metaslab_group_alloc() for small vdevs
When creating a pool on a small vdev (< 1GB) with a low
zfs_vdev_min_ms_count value (e.g., 2), vdev_metaslab_set_size() could
compute a metaslab shift that results in fewer metaslabs than
zfs_vdev_min_ms_count, triggering an assertion failure in
metaslab_group_alloc():
ASSERT3U(mg->mg_vd->vdev_ms_count, >=, 2) failed (1 >= 2)
The root cause is that highbit64(asize / zfs_vdev_min_ms_count) returns
last set bit, which can yield a metaslab size larger than
asize / zfs_vdev_min_ms_count. For example, with a 500MB vdev and
zfs_vdev_min_ms_count=2:
- asize / 2 = 250MB
- highbit64(250MB) = 28, giving 256MB metaslabs
- 500MB / 256MB = 1 metaslab (violates minimum of 2)
Fix by subtracting 1 from highbit64() result to ensure the metaslab
size is at most asize / zfs_vdev_min_ms_count, guaranteeing at least
[22 lines not shown]
[CIR] Accept a union covered only by a bit-field's declared type (#222108)
The size of a union is calculated by its largest member. When that
member is a bit-field, the unit storing it can be narrower than the type
it was declared with, and it is the declared type that accounts for the
union's bytes. BitFieldType gains a query for that declared type, and
the x86_64 union rule reads it rather than the stored size, so `union {
int x : 3; }` and `union { int x : 3; char c; }` now both pass as i32.
A union larger than one eightbyte still needs a member covering it
outright, since the coerce basis skips the entry carrying the
declaration.
Assisted-by: Cursor / claude-opus-5
[AMDGPU][MC] Diagnose bf16 inline constants without op_sel in the ass… (#223035)
…embler
GFX1250 generates a bf16 inline constant in the high half of the
corresponding fp32 inline constant. The VOP1 bf16 opcodes therefore only
read it correctly in the VOP3 encoding with op_sel[0] set:
v_cvt_f32_bf16, v_rcp_bf16, v_sqrt_bf16, v_rsq_bf16, v_log_bf16,
v_exp_bf16, v_sin_bf16, v_cos_bf16 and v_tanh_bf16.
Codegen has applied this workaround since 57a9c4f939e5, but hand-written
assembly and inline asm had no protection at all and silently assembled
to instructions that read the wrong half. Reject those forms in the
assembler.
Only single-source bf16 opcodes are affected. Multi-source and packed
bf16 instructions such as v_fma_mix*_bf16, which also have a scalar bf16
src0, are explicitly excluded.
[3 lines not shown]
[mlir][ROCDL] Carry `arch`'s xnack/sramecc onto the module
`rocdl-attach-target` rejected a target ID that pinned xnack or sramecc,
because `#rocdl.target` feeds a TargetMachine and the backend no longer
accepts those two as subtarget features. Now that the module attributes
exist, migrate them instead of refusing: `TargetInfo` gains
`migrateArchFeaturesToModuleFlags`, which records the settings the target
ID pinned as `rocdl.xnack` / `rocdl.sramecc` on a module, and
`rocdl-attach-target` calls it on each module it attaches to.
A setting the target ID leaves open, or that the GPU does not support, is
left alone rather than written as false: an absent flag means "either",
so writing false would be a different request. That also means an
attribute already on the module survives an `arch` that says nothing
about the feature, while an `arch` that does pin it wins as the more
specific request.
[mlir][AMDGPU] Keep `chipset` as a deprecated alias for `arch`
Renaming the option meant every existing invocation of these passes had
to be updated in lockstep. Accept the old spelling instead: `chipset` on
`convert-amdgpu-to-rocdl`, `convert-gpu-to-rocdl`, `convert-arith-to-amdgpu`,
`convert-math-to-rocdl` and `amdgpu-emulate-atomics`, and `chip` on
`gpu-lower-to-rocdl-pipeline`, which is what each of them was called
before the rename.
`arch` wins whenever it names a target; the alias is consulted only when
`arch` is still at the sentinel that means "no target given", so with
neither given the error still names the unusable default rather than an
empty string, and a stale alias value is reported as itself.
[mlir][AMDGPU] Take an `arch` target ID instead of triple/chip/features
`features` was a general `-mattr` string, which needed a general feature
parser and let callers ask for arbitrary combinations we have no interest
in supporting. In practice the only things anyone sets are the wavefront
size and the xnack/sramecc settings that come off a device query.
Replace `triple`, `chip` and `features` with a single `arch` option that
names the target the way Clang does, parsed by `llvm::AMDGPU::TargetID`
rather than by hand. It accepts
- a processor, with optional target-ID modifiers: `gfx942`,
`gfx942:xnack+`, `gfx9-4-generic`;
- a triple: `amdgpu9.42-amd-amdhsa`;
- a full target ID: `amdcgn-amd-amdhsa--gfx90a:sramecc+:xnack-`, which
is what `rocminfo` prints for a device's ISA, so that output can be
pasted straight in.
Since `chipset=gfx942` becomes `arch=gfx942`, migration is a rename.
[22 lines not shown]
[mlir] Migrate AMDGPU/ROCDL to targets, not chipset versions
**migration tl;dr:** `chipset=` becomes `triple=`, migrate off of
`amdgpu::Chipset` to `ROCDL::TargetInfo`, and eventually change
`gfxXYZ` to `amdgpuX.YZ-amd-amdhsa` in that `triple` argument.
`amdgpu::Chipset` was an awkward hack that was hard to keep up to date
with changes in the compiler/new architectures, and didn't properly
support generic targets (and has been strongly disfavored by the
compiler team).
This PR replaces `amdgpu::Chipset` with `ROCDL::TargetInfo`, a
structure that uses LLVM's TargetParser and the underlying LLVM
features tables to get the real nature of the target being compiled
for.
This also helps MLIR move to
new-style (`-mtriple=amdgpuX.YZ-amd-amdhsa`) over "old
style" (`-mtriple=amdgcn-amd-amdhsa -mcpu=gfxXYZ`) triples.
[40 lines not shown]
[mlir][AMDGPU][NFC] Pre-commit tests for incorrect version checks
There'll be a refactoring from `amdgpu::Chipset` to
`ROCDL::TargetInfo`, thus also moving from chip version checks to
features checks. This commit adds tests for incorrect lowerings that
were allowed by the current code.
- gfx90c is >= gfx90a but stil needs atomic emulation (it doesn't
have buffer fmax and so on).
- gfx90c is also >= gfx90a but has no barrier back-off, so it needs
the inline asm workaround around `s_barrier` that it isn't getting
- gfx908 doesn't have a packed fp16 atomic add but we thought it did
- gfx950 is mistakenly allowing xf32 MFMAs
- gfx1200 is allowing permlane_swap instructions that it doesn't have
- gfx11.7 should be allowing OCP FP8 conversions but isn't on the list
This also cleans up some redundant tests with a --check-prefixes
AI disclosure: Claude found these and wrote the tests.
[2 lines not shown]
[mlir][ROCDL] Add `rocdl.xnack` and `rocdl.sramecc` module attributes
Since 27eeb7370281, the AMDGPU backend takes the xnack
and sramecc target-ID settings from the `amdgpu.xnack` and
`amdgpu.sramecc` module flags instead subtarget features, making the
old usage a hard error.
This commit adds `rocdl.xnack` and `rocdl.sramecc` module attributes
to the discardable attribute list the ROCDL dialect defines in order
to represent these flags and adds translations for them.
Omitting them means to leave these modifiers at
their default "either" state, which isn't the same as setting them to
false.
AI disclosure: Claude wrote this code and I reviewed it and tried to
reword the comments to something better.
