[TargetInstrInfo] Address review: autogenerate greedy-inline-asm-fold.mir checks
Generate the checks with update_mir_test_checks.py so that the spills and
reloads around each folded inline asm are verified, not just the INLINEASM
itself. Fix the "=r" comment to describe the C "+r" operand, note that which
"+rm" pair greedy folds is its own choice, and add a "+&rm" case on an
uninitialized value showing that folding a pair whose tied use is undef
stores to the slot without loading from it.
Assisted-by: Claude Opus 5.5
Test a tie whose def moves when a folded operand is before it
Folding a "+rm" pair before a "+r" pair moves the "+r" pair's def, so
both ends of its tie get new indices. The other tests only moved the
use.
Assisted-by: Claude Opus 5.5
[TargetInstrInfo] Address review: don't fold a subregister that isn't at the start of the slot
A folded operand refers to the start of the stack slot, which holds the
whole register, so a subregister is only there if it's at bit offset 0 on a
little-endian target. Folding x86's sub_8bit_hi read byte 0 instead of byte
1. Folding several operands at once made this worse: two operands holding
different subregisters of one value (e.g. "Qm"(lo), "Qm"(hi)) both read
byte 0.
Check the subregister in mayFoldInlineAsmRegOp, so the spill weights agree
with what folds. A tied def is covered too, since its tied use has the same
subregister.
Assisted-by: Claude Opus 5.5
[TargetInstrInfo] Fix folding inline asm operands next to tied operands
foldInlineAsmMemOperand() swapped a register operand for the target's
memory operands with MachineInstr::removeOperand(), which asserts when a
later operand is tied, because moving it would break the tie. Inline asm
lists every input after every output, so folding any operand that comes
before another tied pair asserted, e.g. an "rm" input followed by the
input of a "+r" operand, or one of two "+rm" operands. Without
assertions, the moved operands kept stale tie indices. Untie the
operands, rebuild the operand list, and re-tie the remaining pairs at
their new positions.
It also gave up when the register appears in more than one operand, e.g.
one value passed to two "rm" operands, which left the greedy allocator
unable to spill that value at all. Fold every such operand into the
stack slot.
Finally, take MayLoad from the folded operands rather than from every
read of the register: a folded def whose tied use is another virtual
[9 lines not shown]
Include STLExtras.h for the range helpers
foldInlineAsmMemOperands uses is_contained, min_element, drop_begin,
and enumerate from STLExtras.h, which the file got only through other
headers.
Assisted-by: Claude Opus 5.5
Move the later folding fixes to their own PRs
1d50fec74e60 added more fixes for folding inline asm register operands
to this PR. They're split into the PRs stacked on this one, one fix
each, to keep this one to the tied operand fix that was reviewed.
This reverts commit 1d50fec74e601c10811a0d58c37a392b2a1ba146.
Assisted-by: Claude Opus 5.5
[CodeGen] Fix more hazards in folding inline asm register operands
Nothing in tree marks an inline asm operand foldable until #229632, so
these were found by a review of the stack rather than by tests:
- The spill weights took a register to be foldable into an asm if any one
of its operands there was foldable, while the folder needs all of them
to be. One value passed to an "rm" and an "r" operand could then get
spilled, not folded, and reloaded into a register the asm didn't have.
Add MachineInstr::mayFoldInlineAsmReg(), true if every operand of a
register in an asm can be folded, and use it in the folder,
CalcSpillWeights, and InlineSpiller.
- The folder now folds every operand of the register, including an undef
use that InlineSpiller leaves out of the operands it passes, which was
left behind still referring to the spilled register.
- InlineSpiller rematerialized a value passed to a foldable operand (e.g.
a constant) into a register right before the asm, which the asm may not
have had. It's spilled and folded instead.
- An asm that can fold a register no longer adds to the register's spill
[18 lines not shown]
[TargetInstrInfo] Address review: don't fold one register of a multi-register operand
A memory operand replaces exactly one register, so return nullptr for a
folded operand whose flag holds more than one, rather than relying on an
assert that release builds drop. Also assert that every folded operand
holds the same register, and say that a caller folding a def whose tied use
is another register must store that register's value into the slot.
Test both the new bail-out and folding a rematerialized load into two "rm"
operands, which takes the fold-a-load path.
Assisted-by: Claude Opus 5.5
[TargetInstrInfo] Address review: write the two-register flag symbolically
Now that the operand count of an inline asm flag can be written in MIR, use
"reguse:GR64 foldable ops:2" instead of an immediate whose register class ID
changes whenever a register class is added. Regenerate the checks, which
now print the five operands of each x86 memory reference as "ops:5".
Assisted-by: Claude Opus 5.5
[MIR] Round-trip the "func" kind of inline asm operand flags
An x86 inline asm call or jmp whose target is a function, given as a "p"
operand or as a "*m" operand that ISel turns into an address, gets a flag
of kind Func. Like a Mem flag, it carries a memory constraint. But
getKindName() returned "mem" for it, and the MIR printer printed the
constraint only for a Mem flag. So this:
call void asm sideeffect "call ${0:P}", "p,~{dirflag},~{fpsr},~{flags}"(ptr @g)
was printed as:
INLINEASM &"call ${0:P}", sideeffect attdialect, mem ops:5, target-flags(x86-plt) @g, 1, $noreg, 0, $noreg, ...
which parses back as a Mem flag with no constraint. Printing that again
hits llvm_unreachable, and the asm printer asserts in PrintMemReference()
instead of emitting "callq g at PLT". MachineInstr::print() also printed the
constraint as a register class (e.g. "mem:GR8_NOREX").
[4 lines not shown]
[MIR] Serialize the operand count of inline asm operand flags (#235184)
An inline asm operand flag is followed by the number of operands that
it says: e.g. two registers for an i128 or an i386 `"A"` output, or the
five operands of an x86 memory reference. The symbolic flag syntax had
no way to write that count, so the printer dropped it and the parser
assumed one. Printed MIR with such a flag didn't parse back correctly:
the operands after the first were read as further flags or as stray operands,
which the machine verifier rejects. Tests had to write such a flag as an
immediate, which holds its register class's TableGen ID: adding a
register class can change the immediate, though the class stays the same.
Print the count as 'ops:N' at the end of the flag when it isn't 1, and
parse it back, e.g.:
INLINEASM &"", attdialect, mem:m ops:5, %stack.0, 1, $noreg, 0, $noreg
INLINEASM &"", attdialect, reguse:GR64 foldable ops:2, %0, %1
A count of 0 is accepted, since the printer emits whatever count the
[16 lines not shown]
[X86][ACEV1][NFC] Take the TOP4MX vvvv source last
The five mixed-precision outer products listed their sources reversed,
because MRMSrcReg takes the EVEX.vvvv operand ahead of the ModR/M one
and the specification numbers them the other way round. MRMSrcReg4VOp3
takes it last, so every operand list now reads in order, as the other
outer products already do. Pairing that form with a trailing immediate
needs the disassembler emitter to accept a fourth operand, which it
already does for MRMSrcReg. The encodings do not change.
[X86][ACEV1] Diagnose a mixed tile model and check the builtins
One palette is in force at a time, so a function cannot configure both
kinds of tile. Report that after selection, when every model has been
recorded, and report a function mixing the two ACE models only once.
Give the AMX tile-zero builtins their own amx-tile spelling, so an ACE
target no longer selects palette 1 for them, and check the scale group
the mixed-precision forms take. The clang tests now pin the tile shape
and alignment rather than the intrinsic name alone.
[X86][ACEV1] Move palette 2 tiles a row at a time
Palette 2 has no TILELOADD or TILESTORED, so a tile reaches memory only
through TILEMOVROW. The casts substituted a zeroed tile for a loaded one
and dropped the stored result, a tile PHI and a tile-to-tile copy still
asked for TILELOADD, and a constant row index went through a GPR.
Synthesise the 16 row moves in each of those places, and select the
immediate form when the index is constant. The -O0 config now leads the
block, since palette 2 writes no shapes and ldtilecfg resets the BSR.
[X86][ACEV1] Record the program model and correct the outer products
Record the program model as the intrinsics are selected, keyed on the
intrinsic as AMX does, rather than from a 66-entry opcode table kept in
step by hand; using both models is now diagnosed, not asserted. Drop the
K dimension the internal outer products never used, and swap the TOP4MX
pseudos' sources, which reached the instruction reversed because it
takes its VVVV operand first. Widen the insert pseudos' index to i32 so
an out-of-range one truncates as for AMX, check it in clang against the
architectural 0-15, and restore the GCCRegNames grouping BSR0 reflowed.
[X86][ACEV1] Give the row and column inserts the tile they write
A tile row or column write keeps the rest of the tile, but the internal
intrinsics took no tile to write into, so X86ExpandPseudo fabricated the
tied source as undef. Nothing linked consecutive inserts, so two tiles
written alternately both got tmm0 and overwrote each other. Add the tile
as an argument, as the outer products take their accumulator.
That same missing operand was why X86LowerAMXType skipped the volatile
tile model on any ACE subtarget, costing plain AMX functions a transform
they need at -O0; volatileTileData already skips ACE tiles per value.
[X86][ACEV1][NFC] Emit the ACE instruction definitions from multiclasses
Most ACE operations need three definitions: the instruction itself, a pseudo
for the user-facing intrinsic that takes an immediate tile ID, and a pseudo
for the internal intrinsic that takes a shape and a tile register so the
register allocator can assign TMMs. All three were written out by hand for
every operation, so the four integer outer products differed only in
mnemonic and prefix, and the twelve row-read pseudos only in the intrinsic.
Emit them from five multiclasses, one per operation shape, deriving the
intrinsic names from the mnemonic as X86InstrAMX.td already does.
[X86][ACEV1] Model the tile row and column writes as read-modify-write
TILEMOVROW and TILEMOVCOL writing into a tile update one row or column and
keep the rest, so the destination is a source as well; the ACE v1
specification encodes operand 1 of all four write forms as REG(r/w). Tie
it. Palette 2 has no TILELOADD, so a reload rebuilds the tile a row at a
time, and without the tie every write but the last looked dead and the
sequence was not valid SSA before register allocation.
Moving a tile also stops asking the subtarget which palette it belongs to.
An ACE target can hold palette 1 tiles, which took the row-by-row route
shaped for palette 2's fixed 16x64 tile. Key it on the programming model,
as the palette byte in the same function already is.
[X86][ACEV1] Give ACE spellings to the shared palette-bearing instructions
ACE configures tile palette 2 and AMX palette 1, so an instruction both
extensions spell the same way must stay distinguishable after ISel. Asking
the subtarget answers wrongly for a function built for an ACE target that
holds only plain AMX code, and silently miscompiled it: a tile loaded
through a vector became a tilezero, dropping the load, and a tile read back
as a vector became poison.
Give the seven shared palette-bearing instructions - TILEZERO, TCVTROWD2PS,
TCVTROWPS2BF16H/L, TCVTROWPS2PHH/L and the TILEMOVROW read form - ACE
intrinsics and pseudos of their own, so the palette is chosen from the
intrinsic while still in IR and, after selection, from the ACE programming
model that the pseudo's opcode records in X86MachineFunctionInfo beside the
AMX one. The read direction of TILEMOVROW also gets a user-facing name,
llvm.x86.acev1.tilemovrowextract, as GCC gives it.
The six row reads take the row index by value in one intrinsic, as GCC's
builtins do, and each has an imm8 encoding alongside the r32 one, so they
[3 lines not shown]
Address review comments
- Rename the ACE tests and directories to acev1 for consistency
- Build acev1 on top of avx10.1 rather than avx10.2
- Fix test encodings and the BSR0 implicit defs
- Make the scale register type consistent with the spec
- Add memory operand tests for bsrmov
- Confirm the ACEV1 ISA implication chain
- Take the outer product ZMM sources by value instead of by register ID,
so they are allocated conventionally, and mark the 512-bit ACE builtins
with RequiredVectorWidth<512>
- Use pseudos for SPILL and RELOAD. Spills use scratch zmm. Don't use
non-ace ISA.
- Fix missing vector r/m in MRMSrcReg4VOp3.
[X86][ACEV1] Address review comments
- Drop the AMX_TILE dependency; under acev1 enable only the AMX
instructions that ACE v1 actually shares.
- Namespace the ACE-invented intrinsics under llvm.x86.acev1.* and leave
the shared AMX-AVX512 ones in the flat llvm.x86.* namespace.
- Sync the public intrinsic and builtin names with GCC, and add the
helper macros from the ACE v1 spec.
- Tighten Sema checking of the outer-product group bits, and add driver,
preprocessor and feature-error test coverage.
- Spell BSR0 literally in the AsmStrings with Defs/Uses instead of
carrying it as a custom operand, which removes the bespoke handling
from X86AsmParser and both instruction printers.
- Drop the unused BSR register class; like TMMCFG, BSR0 needs only its
X86Reg definition and the getReservedRegs entry.
- Adopt the operand-type record naming convention, 't' for a TILE and
'e' for a GR32 index, on both the instructions and their pseudos,
retiring the ad-hoc _REG suffix.
- Use WriteSystem rather than WriteVecLogic for the tile outer products
[6 lines not shown]
[ADT] Thread the ImutAVLFactory free list through the nodes (#244626)
Freed nodes are kept in a vector that grows with the number of freed
nodes. Link them through their now unused left pointers instead,
removing the vector's allocation and growth.
Based on #196534 by Christoph Cullmann.
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: Gabor Horvath <gaborh at apple.com>
Co-authored-by: Christoph Cullmann <cullmann at kde.org>
Co-authored-by: Balázs Benics <benicsbalazs at gmail.com>
[Mips] Don't use trunc.w.fmt for fptosi on MIPS I (#244859)
MIPS I has no trunc.w.fmt. fptosi from float selected trunc.w.s anyway,
since its pattern was marked ISA_MIPS1, and fptosi from double failed
to select.
On MIPS I use cvt.w.fmt with the rounding mode set to round toward
zero, then restore the FCSR. This is the same sequence the assembler
uses for the trunc.w.fmt macro.
cfc1/ctc1 have the same delay as mfc1/mtc1 before MIPS IV, so add them
to the FPU delay slot hazard list. This puts a nop after cfc1 before its
result is used and after ctc1 before the next FPU instruction.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[clang-tidy] Add SARIF output to clang-tidy (#219182)
[clang-tidy] Add SARIF output for clang-tidy diagnostics.
Clang-tidy is a static analysis framework, and should therefore be able
to output results in the Static Analysis Results Interchange Format
(SARIF). This brings clang-tidy in line with clang and
clang-static-analyzer.
Adds an experimental '--experimental-sarif-export=<file>' option to
clang-tidy that writes diagnostics in SARIF format.
Each diagnostic is reported as a SARIF result with its check name,
severity, and location. Notes are emitted as related locations with
their messages, while notes from clang-analyzer checks are emitted as
thread flows so the analyzer path can be followed in SARIF viewers.
To support this, the shared SARIF writer in clang now skips locations
that are invalid or not in a file instead of asserting, allows a message
on related locations, and fixes artifact indices not matching their
position in the artifacts array. The diagnostic level helpers in
[7 lines not shown]