[NVPTX] Honor !atomic.ignore.denormal.mode on atomicrmw fadd
PTX atom.add has a fixed denormal behavior that the program cannot
control: atom.add.f32 flushes denormals on global memory but not on
shared, and atom.add.f16 never flushes. When that disagrees with the
function's denormal mode, the backend expands the atomic into a CAS loop
so the denormal behavior is preserved.
!atomic.ignore.denormal.mode says the denormal behavior of this
particular atomic does not matter, so use the native instruction even
when it disagrees. This is the same thing -nvptx-allow-ftz-atomics does,
except per-instruction instead of per-compilation, which lets a frontend
opt in only the operations it knows about -- notably CUDA's atomicAdd(),
which is defined in terms of atom.add.
Note that -nvptx-allow-ftz-atomics defaults to true, so the new behavior
is only observable with -nvptx-allow-ftz-atomics=false.
Co-authored-by: Artem Belevich <tra at google.com>
[clang][NVPTX] Emit !atomic.ignore.denormal.mode for CUDA atomics
CUDA's atomicAdd() family is defined in terms of PTX atom.add, whose
denormal behavior is fixed by the hardware. Without any annotation the
backend has to assume the function's denormal mode must be honored and
expands these into CAS loops whenever the two disagree. Mark them with
!atomic.ignore.denormal.mode so the native instruction is used.
That covers the __nvvm_atom_*_add_gen_f builtins that atomicAdd(),
atomicAdd_block() and atomicAdd_system() are written in terms of, plus
C11/C++11 atomics under -fatomic-ignore-denormal-mode and the
[[clang::atomic(ignore_denormal_mode)]] attribute, which requires
teaching the NVPTX target about AtomicOptions.
The condition for when the metadata is meaningful is now shared with the
AMDGPU and SPIR-V targets in addAtomicIgnoreDenormalModeMetadata(). It
takes an AllowHalf flag because whether f16 denormals are observable is
target specific: PTX exposes no FTZ control for f16 operations, so
atom.add.f16 never flushes and the opt-in is meaningful there, whereas
[3 lines not shown]
[IR] Generalize !amdgpu.ignore.denormal.mode into !atomic.ignore.denormal.mode
The !amdgpu.ignore.denormal.mode metadata tells the backend that an
atomicrmw fadd need not honor the function's denormal mode, so a native
atomic instruction whose denormal behavior is fixed in hardware may be
used instead of a CAS loop. Nothing about that is AMDGPU specific: NVPTX
has exactly the same problem with atom.add, whose FTZ behavior depends on
the address space and cannot be controlled.
Promote it to a target independent fixed metadata kind,
!atomic.ignore.denormal.mode, and switch the AMDGPU, SPIR-V and OpenMP
producers and consumers over to it. Document it in LangRef, and point
AMDGPUUsage at that description rather than duplicating it.
Existing IR keeps working: AutoUpgrade renames the metadata on atomicrmw
instructions when parsing textual IR and when materializing bitcode. The
upgrade is deliberately scoped to atomicrmw rather than being applied to
every attachment of that name, since that is the only place the metadata
was ever meaningful. Because bitcode can be materialized one function at
[6 lines not shown]
[LV] Move isCandidateForEpilogueVec to LoopVectorizationPlanner.cpp (NFC). (#195268)
isCandidateForEpilogueVectorization (together with its file-static
helper hasUnsupportedHeaderPhiRecipe) does not depend on anything in
LoopVectorize.cpp, move it to LoopVectorizationPlanner.cpp.
PR: https://github.com/llvm/llvm-project/pull/195268
security/vuxml: Document mod_gnutls -- multiple vulnerabilities
CVE-2026-33307 is an out-of-bounds write when mod_gnutls receives a client
certificate chain longer than its buffer, CVE-2026-33308 a missing Key
Purpose check during client certificate verification. Upstream fixed both
on 2026-03-20; in the tree the fix arrives with www/mod_gnutls 0.13.0.
Sponsored by: Netzkommune GmbH
[VPlan] Split legalizeAndOptimizeIVs in 2 phases. (#217765)
Split the transform into 2 phases:
1. narrow all users of all IVs
2. replace wide IVs if all users are scalar.
Together with removing the old, wide recipes, this allows us to catch
slightly more cases.
PR: https://github.com/llvm/llvm-project/pull/217765
[RISCV] Cost i64 accumulator for Zvdot4a8i partial reductions
Now that `lowerPARTIAL_REDUCE_MLA` can lower an i64 accumulator with
i8 inputs using the dot-product instructions, teach the cost model
to price it so the vectorizer will form it.
- `getPartialReductionCost` accepts an i64 accumulator (reduction
factor 8) and prices the `vdot4a*` plus the reduce-and-accumulate
for both lowering shapes:
- reduce the i32 partial sums with a `vadd.vv` and widen into the
accumulator with a `vwadd.wv` (fixed-length vectors additionally
extract the high i32 subvector with a `vslidedown`), and
- for a single-vector scalable accumulator, whose i32 subvectors are
a fractional LMUL, widen to i64 first and reduce/accumulate with
two i64 `vadd.vv`.
The i64 case requires a wide enough VF (LMUL) to reach the scale-8 factor,
matching how AArch64 only forms it under SVE.
[6 lines not shown]
e1000: Serialize 82579 CSR writes with the Management Engine
The 82579 PCIm2PCI arbiter can acknowledge a host MAC CSR write while
the Management Engine is accessing another CSR. The host write can be
lost; subsequent target accesses may no longer be claimed by the MAC and
can hang the system.
For 82579 controllers with valid management firmware, wait for the ME
CSR access indication before every MAC CSR write. Keep the wait bounded
and use DELAY because writes occur in interrupt and datapath contexts.
Verify every transmit and receive tail write. If a tail does not hold
the requested value, disable its datapath direction and request a full
iflib reset.
Keep the ordinary register-write path as a direct MMIO write behind a
predicted per-device gate. Contain the wait and tail recovery in the
82579 slow path rather than adding tail-specific accessors and state to
the rest of the e1000 family.
[11 lines not shown]
[lldb][test] Disable lldb-dap watchpoint test on Arm Linux (#218360)
Added by #215228. The test assertions are fine, but when the program
continues at the end, it faults.
More details in https://github.com/llvm/llvm-project/issues/217961.
[AMDGPU] Combine redundant ballot intrinsic calls
Suppose there is a loop where there is a call to @llvm.amdgcn.ballot,
which maps to an instruction involving the exec mask as an operand. This
instruction duplicates if the loop is unrolled. With a higher number of
unrolled iterations, the code bloats with such redundant instructions
with $exec as there is no middle-end/backend pass which could combine
such instructions in a uniform CFG.
This patch introduces a transform in AMDGPUUniformIntrinsicCombine to
combine redundant calls to @llvm.amdgcn.ballot, to mitigate this issue.
The approach is to walk over the dominator tree and collect all calls to
@llvm.amdgcn.ballot. Map the result type and condition to the calls, to
avoid combining calls of different kinds. Calls A and B can be combined
into A iff:
- A and B are identical
- A dominates B
- all paths from A to B are uniform and exec-invariant.
[2 lines not shown]
[MergeFunc] Preserve observable function pointer identity (#213604)
Fixes #213206.
MergeFunc can replace one function with a forwarding thunk to another.
FunctionComparator treated self-references as equal in every context, so
functions that observe their own address (e.g. icmp %p, @g) could be
merged incorrectly.
Compare ordinary function-pointer uses as normal global values. Treat
self-references as equal only for corresponding call targets and
blockaddress.
AI assistance was used while preparing this change.
[lldb] Handle 0 size sites in StopPointSiteList::FindInRange (#217919)
Fixes #205120
In which due to delayed breakpoints, a breakpoint that would become an
external breakpoint later (meaning managed by the debug server) was
temporarily stored as a software breakpoint (which is managed by lldb)
with a zero size breakpoint site. That zero site site tripped an
assertion when you tried to write over the site.
To fix this, I've explicitly ignored zero size sites in FindInRange by
defining them as never overlapping. FindInRange is only used for
patching reads and writes, so I think this is safe to do. I have
documented this in the docstring.
I considered adding a breakpoint type "uncommitted", but software
breakpoints are actually handled in the most conservative manner (reads
and writes are always patched). So I think as a default it's fine (also
I don't want to go and audit all the places that use that enum and end
[4 lines not shown]
