[mlir][AMDGPU] Add preferAgpr hint to MFMA operations
The AMDGPU backend accepts the `!amdgpu.mfma !{!"agpr"}` instruction
metadata on calls to the `llvm.amdgcn.mfma.*` intrinsics (including
`llvm.amdgcn.mfma.scale.*`). The metadata is a hint that asks the backend
to allocate the accumulator input (C) and the result (D) of the MFMA to
AGPRs. The backend can ignore the hint, and dropping it does not change
program behavior.
Add an optional `preferAgpr` unit attribute to the ROCDL MFMA intrinsic
operations, including the scaled MFMA operations, and to `amdgpu.mfma` and
`amdgpu.scaled_mfma`. When the attribute is present, the translation to
LLVM IR attaches `!amdgpu.mfma !{!"agpr"}` to the intrinsic call, and the
AMDGPUToROCDL conversion copies the attribute to the ROCDL operation. When
the attribute is absent, the printed operations and the translated LLVM IR
are the same as before.
The SMFMAC operations do not get the attribute because the backend ignores
the hint on `llvm.amdgcn.smfmac.*`.
[5 lines not shown]
[AMDGPU] Keep metadata when shrinking scaled MFMA fp4/fp6 operands
InstCombine narrows the A/B operands of llvm.amdgcn.mfma.scale.*.f8f6f4
for fp4/fp6 formats by creating a new call, but did not copy the old
call's metadata. Copy it, as the other rewrites in this file do, so
hints such as !amdgpu.mfma survive.
[AMDGPU][SelectionDAG] Select the MFMA AGPR form from !amdgpu.mfma metadata
Problem
- Frontends (Triton) pin MFMA accumulators to AGPRs with "=a" inline asm.
- The asm forces VGPR->AGPR copies around every MFMA (v_accvgpr_mov
+ s_nop inside loops), and selection has no way to ask for the AGPR
form per instruction: it is chosen per function.
Idea: a droppable hint on the MFMA call, read when the form is selected.
IR call @llvm.amdgcn.mfma.*(...), !amdgpu.mfma !{!"agpr"}
| SelectionDAGBuilder: TLI.getTargetIntrinsicHint(I)
DAG INTRINSIC_WO_CHAIN --SDEI side table: TargetHint
| CanUseAGPR_MAI predicate
MIR V_MFMA_..._e64 (AGPR form): C/D are AReg, no copies
How
1. TargetLowering::getTargetIntrinsicHint returns metadata from an
intrinsic call; SelectionDAGBuilder stores it on the node in the
[34 lines not shown]
AMDGPU: Fold inline immediates in peephole-opt (#242952)
Currently the AMDGPU implementation of foldImmediate handles a small
set of special cases that require rewriting the instruction opcode
(COPY -> mov, and fma-like to fmac-like). Most general immediate folding
is handled in SIFoldOperands. Teach PeepholeOpt to fold all inline
immediates into target instructions which should always be an
improvement.
This is to help alleviate some phase ordering problems in future changes.
I've also never liked how SIFoldOperands is written and it could use a
rewrite, and this will alleviate some of its responsibilties. It will
always be necessary since some operand folds require additional context
that foldImmediate won't have (namely, we have to make contextually
aware tradeoffs for which operands are worth folding to respect constant bus
restrictions, considering other users of the values).
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
AMDGPU: Clear stale kills when merging v_mov_b16 pairs
SIPostRA16BitMovFolding can insert the merged 32-bit instruction after
an instruction that kills the first mov's source, which then reads a
register the verifier considers dead:
$vgpr7_lo16 = V_MOV_B16_t16_e32 $vgpr5_lo16
$vgpr16 = V_PERM_B32_e64 killed $vgpr5, $vgpr6, ...
$vgpr7_hi16 = V_MOV_B16_t16_e32 killed $vgpr6_lo16
=>
$vgpr16 = V_PERM_B32_e64 killed $vgpr5, $vgpr6, ...
$vgpr7 = V_PERM_B32_e64 $vgpr6, $vgpr5, ...
Clear kill flags of the first mov's source between the first mov and
the insertion point.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
emulators/mame: Enable TOOLS option by default
Enable building tools by default, as they add little to the build time.
This avoids requiring users who need the tools to rebuild MAME.
PR: 300705
Reported by: Benoit Felix <benoit.felix at outlook.com>
Approved by: thierry (mentor, implicit)
IR: Check for overlap before copying in AttributeSet::removeAttributes
removeAttributes copied the whole set into an AttrBuilder to check if
the mask overlaps it, which it usually doesn't. Check the attributes
of directly with the mask and only build the AttrBuilder if something
needs to be removed.
callgrind instructions, opt -O3: kimwitu++ k.bc -0.16%, SPASS cnf.bc
-0.08%. CTMark instructions:u: ThinLTO -0.02% (kimwitu++ -0.06%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[orc-rt] Rename SymbolLookupSet to RemoteSymbolLookupSet. NFC. (#242950)
The std::string-based set is only used for lookup requests that arrive
from the controller. Renaming it frees SymbolLookupSet for a set of
interned (StringPool::Ptr) names, and matches the existing names on the
LLVM side, where llvm::orc::SymbolLookupSet holds interned names and
llvm::orc::RemoteSymbolLookupSet holds plain strings.
The SPS tag SPSSymbolLookupSet is unchanged: it names the wire format,
which LLVM uses for both kinds of set.
AutoUpgrade: Skip return values and arguments without attributes
UpgradeFunctionAttributes built the mask of type incompatible
attributes and tried to remove them from all arg and return positions,
even if there were no attributes.
callgrind instructions, opt -O3: kimwitu++ k.bc -0.05%, SPASS cnf.bc
-0.02%. CTMark instructions:u: ThinLTO -0.02%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Fold inline immediates in peephole-opt
Currently the AMDGPU implementation of foldImmediate handles a small
set of special cases that require rewriting the instruction opcode
(COPY -> mov, and fma-like to fmac-like). Most general immediate folding
is handled in SIFoldOperands. Teach PeepholeOpt to fold all inline
immediates into target instructions which should always be an improvement.
This is to help alleviate some phase ordering problems in future changes.
I've also never liked how SIFoldOperands is written and it could use a
rewrite, and this will alleviate some of its responsibilties. It will
always be necessary since some operand folds require additional context
that foldImmediate won't have (namely, we have to make contextually aware
tradeoffs for which operands are worth folding to respect constant bus
restrictions, considering other users of the values).
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
tcp/rack: reject wma_divisor values <= 0
The net.inet.tcp.rack.measure.wma_divisor sysctl is the divisor
of the goodput weighted moving average used when the dynamic
goodput multiplier is enabled. Setting it to 0 panics the kernel.
Make the variable and the sysctl unsigned, reject 0 and values
above INT32_MAX.
MFC after: 1 month
Found with: Claude Code Opus 5.5
Reviewed by: tuexen, glebius, markj (early version)
Differential Revision: https://reviews.freebsd.org/D60759
(cherry picked from commit 7668705cb9837d4c3a6db9b3434b3a522220c8ea)
[X86][ACEV1][NFC] Emit the ACE instruction definitions from multiclasses
Most ACE operations need three definitions: the instruction itself, a pseudo
for the user-facing intrinsic that takes an immediate tile ID, and a pseudo
for the internal intrinsic that takes a shape and a tile register so the
register allocator can assign TMMs. All three were written out by hand for
every operation, so the four integer outer products differed only in
mnemonic and prefix, and the twelve row-read pseudos only in the intrinsic.
Emit them from five multiclasses, one per operation shape, deriving the
intrinsic names from the mnemonic as X86InstrAMX.td already does.
[clang-tidy][docs] Rewrite modernize check docs to Markdown [1/4] (#232207)
Tracking issue: #201242
See the [migration guide] for more information.
[migration guide]:
https://llvm.org/docs/SphinxQuickstartTemplate.html#markdown-migration-guidelines
This is the first part of rewriting check documentations in modernize module from reST to MyST Markdown.
AI Usage: This was prepared with rnk's fork of rst2myst and GPT6-assisted (Codex) cleanup.
I manually verified that the documentation renders as expected.
Preview site:
https://broken.life/llvm-staging/modernize-markdown-port/index.html
Co-authored-by: EugeneZelenko <eugene.zelenko at gmail.com>
[orc-rt] Move SymbolLookupResult into its own header. NFC. (#242948)
Splits SymbolLookupResult out of SymbolLookupSet.h, along with its SPS
serialization (now SPSSymbolLookupResult.h) and its unit tests. This
prepares for adding an interned-string counterpart to SymbolLookupSet:
both kinds of set will share the result type.
Merge commit 72a20b8e2987 from llvm git (by Gabriel Baraldi):
[SLPVectorizer] Check std::optional coming out of getPointersDiff (#175784)
Fixes https://github.com/llvm/llvm-project/issues/175768
There are other unchecked uses std::optional in this pass but I couldn't
figure out a test that triggers them
This fixes a clang crash in the SLP vectorizer, hit when building
audio/csound on powerpc64le. The hardened libc++ traps on the disengaged
optional, so clang dies with exit code 133. The second hunk is adapted to
llvm 21, where analyzeConstantStrideCandidate() is still isStridedLoad().
Approved by: re (cperciva)
Reviewed by: dim
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D61354
[2 lines not shown]
[clang-tidy][docs] Rename modernize check docs to Markdown [1/4] (#232206)
Tracking issue: #201242
See the [migration guide] for more information.
[migration guide]:
https://llvm.org/docs/SphinxQuickstartTemplate.html#markdown-migration-guidelines
This is the first part of the mechanical rename of the check
documentations in modernize module.
The rewrite is provided by the next PR in this stack.
AMDGPU: Fold inline immediates in peephole-opt
Currently the AMDGPU implementation of foldImmediate handles a small
set of special cases that require rewriting the instruction opcode
(COPY -> mov, and fma-like to fmac-like). Most general immediate folding
is handled in SIFoldOperands. Teach PeepholeOpt to fold all inline
immediates into target instructions which should always be an improvement.
This is to help alleviate some phase ordering problems in future changes.
I've also never liked how SIFoldOperands is written and it could use a
rewrite, and this will alleviate some of its responsibilties. It will
always be necessary since some operand folds require additional context
that foldImmediate won't have (namely, we have to make contextually aware
tradeoffs for which operands are worth folding to respect constant bus
restrictions, considering other users of the values).
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>