[RISCV] Declare command line options in TableGen (#230014)
Move the cl::opts of RISCVCodeGen into RISCVOptions.td, and those of
RISCVDesc and RISCVAsmParser into MCTargetDesc/RISCVMCOptions.td,
registered by LLVMInitializeRISCVTargetMC(). RISCVTargetMachine holds
`const RISCVOptions &CLOpts` and RISCVSubtarget copies it;
RISCVAsmBackend, RISCVInstPrinter, and RISCVTargetStreamer hold `const
RISCVMCOptions &CLOpts`. -riscv-rvv-regalloc (RegisterPassParser) stays
cl::opt.
`llvm-objdump -M emit-x8-as-fp` now sets a file-static flag instead of
writing the cl::opt.
-riscv-br-merging-base-cost now takes effect when given once; it
previously required getNumOccurrences() > 1.
Aided by Opus 5.5
[AMDGPU] Fold add of a variable into a zero dot accumulator
When the dot intrinsic has a zero accumulator, no clamp and a single
add user, fold the add operand into the accumulator:
```llvm
%dot = call i32 @llvm.amdgcn.sdot4(i32 %a, i32 %b, i32 0, i1 false)
%r = add i32 %dot, %x
=>
%r = call i32 @llvm.amdgcn.sdot4(i32 %a, i32 %b, i32 %x, i1 false)
```
If %x is defined after the dot in the same block, the dot is moved
down to the add. The fold is skipped across blocks to avoid sinking
the dot into a loop.
[Clang][AMDGPU] Use unsigned int for tensor builtin D# groups (#230714)
The D# tensor descriptor groups of __builtin_amdgcn_tensor_load_to_lds
and __builtin_amdgcn_tensor_store_from_lds are bit fields, not signed
values. D0 was already declared as a vector of unsigned int; D1 through
D4 are now consistent with it.
OpenCL does not allow lax vector conversions, so the tests are updated
to pass unsigned vectors. The generated IR is unchanged.
Reference: https://github.com/llvm/llvm-project/pull/193310
Co-authored-by: Claude Opus 5 <noreply at anthropic.com>
[lldb][docs] Use project-local documentation links
Replace same-project absolute URLs with relative Markdown links and Sphinx
cross-references so local and archived documentation stays self-contained.
Update generated Python API docstrings accordingly, and enable the absolute
link check for the LLDB docs to keep it that way.
[docs] Enable absolute documentation link checks
Configure the LLVM documentation URL prefixes so the Sphinx build rejects
absolute links to documents in the same project. Clang is already configured.
Part of #214861
[clang][Parse] Delay template-id destruction in NTTP default arguments (#230513)
There was a UAF when the lambda appears within NTTP default arguments.
Co-authored-by: Emery Conrad <emery.conrad at chicagotrading.com>
Co-authored-by: Sebastian Schwartz <sebastian.schwartz at chicagotrading.com>
Co-authored-by: Emery Conrad <emery.conrad at chicagotrading.com>
Assisted-by: Claude Code (claude-opus-5-5)
[CIR] Add bytecode encodings for attributes (#229586)
Without a BytecodeDialectInterface, MLIR encodes a dialect's attributes
through their assembly format, embedding the printed text in the
bytecode. This adds native encodings for 27 CIR attributes (constants,
constant initializers, global views, method/data-member pointers, C++
special-member attributes, and small metadata attributes), modelled on
the LLVM dialect's bytecode support.
On attribute-dense modules this shrinks the bytecode ~30-40% (8.6KB to
5.4KB across the new tests). The room ahead is huge, emitting bytecode
for CIRGenModule.cpp (441MB of CIR text) still exceeds 22GB RSS and 30
minutes with these encodings (64GB and 45 minutes without), since types
keep using the assembly fallback. This is paving towards selfhosting
with bytecode.
Coverage is partial by design: anything not listed keeps using the
assembly-format fallback, exactly as before. Types keep the fallback too
until the type encodings land separately.
[3 lines not shown]
[scudo] Check quarantine batch bounds in release builds (#230247)
[Attacking Scudo's Quarantine, section 3: Write Where
Ptr](https://un1fuzz.github.io/articles/quarantine_attack.html#a3)
describes corrupting a quarantine batch's `Count` so that enqueue writes
the freed pointer outside the batch. The original [PoC and exploit code
are
here](https://github.com/un1fuzz/scudo_research/tree/main/quarantine_arbitrary_return).
`push_back()` currently guards its index with a debug-only check; an
oversized count also bypasses enqueue's full-batch equality check.
Make that bound a release-build check. Also validate both counts before
deciding whether batches can merge, enforce merge capacity in
production, and check the count before shuffling (which precedes
recycling). The capacity comparison uses subtraction after validating
both operands, avoiding corrupted-count addition wrapping around.
This closes the out-of-bounds enqueue primitive described in section 3.
It does not address section 2's Double Return attack using in-range
[23 lines not shown]
[CIR][NFC] Add missing NYI handling for some x86 builtins (#230609)
While doing a recent code review, I noticed that there were some x86
builtins that were incorrectly falling through to code that handles
builtins below them in a switch. This change adds an errorNYI diagnostic
rather than falling through.
AMDGPU: Index loads by workitem id in tests shared with r600 (#230667)
These tests relied on -amdgpu-scalarize-global-loads=false to select
vector loads from uniform pointer arguments. They also have r600 run
lines, so keep the kernels and index the input pointers by the workitem
id instead. Also fix shl_v2i16 not using its computed pointers, and
v_shl_32_i64 using the workgroup id. Also add some uniform variants
of some cases.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
fix(AMDGPU): guard OR folds with shared conditions
A shared uniform condition still needs materializing after folding a
divergent OR to a select, and sharing can introduce an extra SCC
conversion. The extension's single-use check does not prevent this
code-size regression.
Check the condition's uses and divergence as well. Add regression and
control cases, and consolidate the boolean OR tests in or.ll.
[CIR] Pass x87 long double vectors on x86_64 (#230302)
Vectors of x87 long double now go through x86_64 calling-convention
lowering instead of hitting NYI. The ABI library sizes their elements at
128 bits like clang does, so the signatures match classic codegen.
Unions are still moved as a value of their storage type, and a long
double stores only 10 of its 16 bytes. So a union holding an x87 value
next to another member stays NYI unless it's a plain long double and the
other members fit in those 10 bytes. That also stops a silent miscompile
of unions like `union { long double ld; char c[16]; }`.
Assisted-by: Cursor / Claude Opus 5.5
[RISCV] Reassociate add (add X, (ext Y)), (ext Z) -> add (add (ext Y), (ext Z)), X (#230657)
If Y and Z are extended from the same bitwidth, then we can reassociate
it so they are in the same inner add.
This then allows combineBinOpOfExt to kick in and narrow the inner add
to vwadd.vv. The motivating case is the sad_* kernels in x264.
Reassociating them allows more arithmetic to stay in a smaller LMUL and
takes up to 40% less cycles on sad_16x16 on the K3. This gives a ~3.1%
speedup on 525.x264_r overall.
AMDGPU: Index loads by workitem id in tests shared with r600
These tests relied on -amdgpu-scalarize-global-loads=false to select
vector loads from uniform pointer arguments. They also have r600 run
lines, so keep the kernels and index the input pointers by the workitem
id instead. Also fix shl_v2i16 not using its computed pointers, and
v_shl_32_i64 using the workgroup id. Also add some uniform variants
of some cases.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
perf(AMDGPU): fold extended boolean OR to select
Require one-use extensions to avoid replacing an e32 OR with an e64
cndmask while the extended value is still needed elsewhere.
[AMDGPU] Fix missed WMMA C-operand co-exec hazard (#226503)
The gfx1250 WMMA co-execution hazard check treats only A, B and the
SWMMAC index as registers the in-flight MMA still reads. C (src2 of a
non-SWMMAC WMMA) is missing, so a VALU scheduled into the MMA's shadow
can clobber C and the MMA consumes the new value.
This is latent while C is tied to vdst, since the existing D check then
covers it. It miscompiles where the tie does not hold: for
v_wmma_bf16f32_16x16x32_bf16, whose D is narrower than C, and for the
_threeaddr form of any WMMA.
HipStdPar: Don't reprocess math library functions as intrinsics (#230603)
All uses were already replaced so this only inserted unused nonsense
declarations by replacing the first 4 characters of the function name
with "__hipstdpar". For example acosh would be replaced with
__hipstdparh.
Co-authored-by: Claude <noreply at anthropic.com>
fix(AMDGPU): fold AND of any-extended booleans
Demanded-bits simplification can turn a boolean sign extension into an
any extension before the target AND combine. Choose sign extension for
the undefined high bits so the AND still folds to a select.
Cover reduced demand, swapped operands, and zero extension. Restore the
fold in setcc-multiple-use after symmetric demanded-bits simplification.
[Mach-O] Parallelize ICF's section sort. NFC (#230269)
Port of #223216 (ELF) and #229849 (COFF) to Mach-O. Replace the
single-threaded llvm::stable_sort of the ICF inputs with a parallelSort
over packed 64-bit keys, then gather the inputs in key order.
The key is icfEqClass[0] in the high 32 bits and the input index in the
low bits. With --icf=safe_thunks, bit 31 is set for inputs that are not
keepUnique, so keepUnique inputs still come first within each class. The
index breaks remaining ties by original position, so the resulting order
is exactly what stable_sort produced and the output is unchanged.
[SelectionDAG] avoid rounding exact f32->bf16 (STRICT_)FP_ROUND
For f32-to-bf16, DAGTypeLegalizer::SoftPromoteHalfRes_FP_ROUND currently
ignores the Trunc flag and unconditionally lowers ISD::FP_ROUND to
ISD::FP_TO_BF16 as follows:
- On X86 targets with +avx512bf16 or +avxneconvert, ISD::FP_TO_BF16
lowers to vcvtneps2bf16, which is lossy (it flushes subnormals to
zero and quietens sNaNs.)
- On targets without hardware bf16 conversion instructions, it performs
rounding via a runtime libcall (e.g. __truncsfbf2) or a software
rounding sequence (which can also be lossy, e.g. quieting sNaNs).
The above is unnecessary and lossy; bitcasting to i32 and extracting
the upper 16 bits is cheaper and lossless. Do that.
Tested:
[11 lines not shown]
fix(SelectionDAG): simplify commuted demanded bits
AND/OR demanded-bit simplification uses RHS known bits to simplify the
LHS, but does not retry the RHS using LHS known bits, making
optimizations depend on operand order.
Retry the RHS when the LHS reduces its demanded bits. Add AArch64
and AMDGPU codegen coverage.
[X86] Keep upper-16-bit f32 extractions in XMM for f16/v8i16
Teach combineBitcast and combineVectorInsert to keep this upper-16-bit
f32 extraction in XMM registers rather than bouncing through a GPR.
This paves the way for the following change in SelectionDAG:
https://github.com/llvm/llvm-project/pull/230557
