[mlir][Vector] Reuse args buffers in WarpOpToScfIfPattern (#204311)
Args buffers are unused after lane 0 reads them inside the scf.if (the
Step 3 barrier fences other lanes), so reuse their storage for yielded
values via `memref.subview` when sizes are compatible. Reduces shared
memory footprint with no runtime overhead.
[RISCV] Add VeeR EH1 scheduling model and tuning (#240913)
VeeR EH1 is a 9-stage, 2-wide in-order core with two ALUs, one
load/store unit and one pipelined multiplier. Load and multiply results
reach an ALU consumer after 3 cycles, a branch can issue in the same
cycle as the load or multiply it depends on (it executes in the EX4
secondary ALU), and the divider stalls decode until it finishes. Checked
against cycle-accurate simulation of the v1.9 RTL.
Use the model for -mcpu=chipsalliance-veer-eh1, with tuning from the
same simulation: the RISC-V unroll preferences (TuneNoDefaultUnroll), no
load clustering (no data cache, one load/store unit), post-RA
scheduling, and a minimum jump-table size of 10 (a branch tree is
cheaper below that).
Assisted-by: Claude
[NVPTX] Only run ptxas on clz-popc-known-bits.ll if it supports sm_70 (#242961)
ptxas from CUDA 13 no longer supports sm_70, so the RUN line added in
#237793 fails on the NVPTX buildbots. Guard it the same way as other
sm_70 tests.
Assisted-by: Claude Code (Claude Opus 5.5)
[mlir] [arith] Combine op for select (#234818)
Fold select(c, F(a0, a1), F(b0, b1)) into F(select(c, a0, b0), select(c,
a1, b1))
Correctness provement:
Let's discuss the cases ignoring poison first.
As F is elementwise, we have
F(a1, a2, ..., a_n)[p] = F(a1[p], a2[p], ..., a_n[p])
then for
L = select(M, F(a0, a1, ..., a_n), F(b0, b1, ..., b_n))
R = F(select(M, a0, b0), select(M, a1, b1), ....)
for any position p, we have:
[35 lines not shown]
[clang-tidy] Add bugprone-macro-condition check (#210768)
Warns about inconsistent macro usage in preprocessor conditions.
#define USE_FOO 0
#ifdef USE_FOO
// ...no preprocessor directives testing the value of USE_FOO
#endif
Here USE_FOO is defined to a value (and furthermore defined to evaluate
to false) but the preprocessor condition only checks for the macro being
defined.
Fixes #27438
[orc-rt] Add SymbolLookupSet for interned symbol names (#242969)
SymbolLookupSet holds symbol names as StringPool::Ptrs, so building a
set doesn't copy the name strings. It has the same interface as
RemoteSymbolLookupSet, which holds plain std::strings.
sps::lookupInJITDylib now takes a SymbolLookupSet. To support this:
- StringPool::Ptr and WeakPtr can be serialized to SPSString, which
writes the string's value. Only serialization is supported:
deserializing into an interned string would need a StringPool, and SPS
deserialization has no way to obtain one.
- SymbolLookupSet can be serialized (but not deserialized) as an
SPSSymbolLookupSet. The wire format doesn't change, so controllers are
unaffected.
- SPSRemoteSymbolLookupSet.h is renamed to SPSSymbolLookupSet.h, after
the tag it defines. It now holds the traits for both kinds of set.
[libc][GPU] Enable scalbn/ldexp, rand, remainder for SPIR-V (#242212)
This PR is a follow up of
https://github.com/llvm/llvm-project/pull/236324 to fix more build
issues for SPIR-V compiler-rt builtin tests. "Undefined symbols" from
libc and libm still lead to a bunch of build failures such as rand,
srand, scalbn/f, remainder/f. Before the fix, building results:
Total Discovered Tests: 275
Unsupported: 125 (45.45%)
Passed : 128 (46.55%)
Failed : 22 (8.00%)
After the fix:
Total Discovered Tests: 275
Unsupported: 125 (45.45%)
Passed : 136 (49.45%)
Failed : 14 (5.09%)
8 more tests can build pass now.
[8 lines not shown]
[llvm-exegesis] Parse library options into a context (#242680)
Parse the command line into a cl::LibraryOptionsContext instead of the
global option values. LLVMState::create takes the context and points
the target machine's MCOptions.LibraryOptions at it. The assembler's
LLVMContext and the disassembler's MCAsmInfo read from it as well.
LLM-aided
AtomicExpand: Cast the xchg from an expanded atomic store to integer
expandAtomicStoreToXChg created the atomicrmw xchg and expanded it
directly, skipping the cast to integer that shouldCastAtomicRMWIInIR
requests for other xchg instructions. A vector store expanded this way
reached the LL/SC expansion with a vector type, which asserted on ARM.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir][AMDGPU] Add preferAgpr hint to MFMA operations
The AMDGPU backend accepts the `!amdgpu.mfma !{!"agpr"}` instruction
metadata on calls to the `llvm.amdgcn.mfma.*` intrinsics (including
`llvm.amdgcn.mfma.scale.*`). The metadata is a hint that asks the backend
to allocate the accumulator input (C) and the result (D) of the MFMA to
AGPRs. The backend can ignore the hint, and dropping it does not change
program behavior.
Add an optional `preferAgpr` unit attribute to the ROCDL MFMA intrinsic
operations, including the scaled MFMA operations, and to `amdgpu.mfma` and
`amdgpu.scaled_mfma`. When the attribute is present, the translation to
LLVM IR attaches `!amdgpu.mfma !{!"agpr"}` to the intrinsic call, and the
AMDGPUToROCDL conversion copies the attribute to the ROCDL operation. When
the attribute is absent, the printed operations and the translated LLVM IR
are the same as before.
The SMFMAC operations do not get the attribute because the backend ignores
the hint on `llvm.amdgcn.smfmac.*`.
[5 lines not shown]
[AMDGPU] Keep metadata when shrinking scaled MFMA fp4/fp6 operands
InstCombine narrows the A/B operands of llvm.amdgcn.mfma.scale.*.f8f6f4
for fp4/fp6 formats by creating a new call, but did not copy the old
call's metadata. Copy it, as the other rewrites in this file do, so
hints such as !amdgpu.mfma survive.
[AMDGPU][SelectionDAG] Select the MFMA AGPR form from !amdgpu.mfma metadata
Problem
- Frontends (Triton) pin MFMA accumulators to AGPRs with "=a" inline asm.
- The asm forces VGPR->AGPR copies around every MFMA (v_accvgpr_mov
+ s_nop inside loops), and selection has no way to ask for the AGPR
form per instruction: it is chosen per function.
Idea: a droppable hint on the MFMA call, read when the form is selected.
IR call @llvm.amdgcn.mfma.*(...), !amdgpu.mfma !{!"agpr"}
| SelectionDAGBuilder: TLI.getTargetIntrinsicHint(I)
DAG INTRINSIC_WO_CHAIN --SDEI side table: TargetHint
| CanUseAGPR_MAI predicate
MIR V_MFMA_..._e64 (AGPR form): C/D are AReg, no copies
How
1. TargetLowering::getTargetIntrinsicHint returns metadata from an
intrinsic call; SelectionDAGBuilder stores it on the node in the
[34 lines not shown]
AMDGPU: Fold inline immediates in peephole-opt (#242952)
Currently the AMDGPU implementation of foldImmediate handles a small
set of special cases that require rewriting the instruction opcode
(COPY -> mov, and fma-like to fmac-like). Most general immediate folding
is handled in SIFoldOperands. Teach PeepholeOpt to fold all inline
immediates into target instructions which should always be an
improvement.
This is to help alleviate some phase ordering problems in future changes.
I've also never liked how SIFoldOperands is written and it could use a
rewrite, and this will alleviate some of its responsibilties. It will
always be necessary since some operand folds require additional context
that foldImmediate won't have (namely, we have to make contextually
aware tradeoffs for which operands are worth folding to respect constant bus
restrictions, considering other users of the values).
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
AMDGPU: Clear stale kills when merging v_mov_b16 pairs
SIPostRA16BitMovFolding can insert the merged 32-bit instruction after
an instruction that kills the first mov's source, which then reads a
register the verifier considers dead:
$vgpr7_lo16 = V_MOV_B16_t16_e32 $vgpr5_lo16
$vgpr16 = V_PERM_B32_e64 killed $vgpr5, $vgpr6, ...
$vgpr7_hi16 = V_MOV_B16_t16_e32 killed $vgpr6_lo16
=>
$vgpr16 = V_PERM_B32_e64 killed $vgpr5, $vgpr6, ...
$vgpr7 = V_PERM_B32_e64 $vgpr6, $vgpr5, ...
Clear kill flags of the first mov's source between the first mov and
the insertion point.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
IR: Check for overlap before copying in AttributeSet::removeAttributes
removeAttributes copied the whole set into an AttrBuilder to check if
the mask overlaps it, which it usually doesn't. Check the attributes
of directly with the mask and only build the AttrBuilder if something
needs to be removed.
callgrind instructions, opt -O3: kimwitu++ k.bc -0.16%, SPASS cnf.bc
-0.08%. CTMark instructions:u: ThinLTO -0.02% (kimwitu++ -0.06%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[orc-rt] Rename SymbolLookupSet to RemoteSymbolLookupSet. NFC. (#242950)
The std::string-based set is only used for lookup requests that arrive
from the controller. Renaming it frees SymbolLookupSet for a set of
interned (StringPool::Ptr) names, and matches the existing names on the
LLVM side, where llvm::orc::SymbolLookupSet holds interned names and
llvm::orc::RemoteSymbolLookupSet holds plain strings.
The SPS tag SPSSymbolLookupSet is unchanged: it names the wire format,
which LLVM uses for both kinds of set.
AutoUpgrade: Skip return values and arguments without attributes
UpgradeFunctionAttributes built the mask of type incompatible
attributes and tried to remove them from all arg and return positions,
even if there were no attributes.
callgrind instructions, opt -O3: kimwitu++ k.bc -0.05%, SPASS cnf.bc
-0.02%. CTMark instructions:u: ThinLTO -0.02%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>