[mlir][LLVM] Use a disjoint scope domain when inlining noalias
This matches recent changes to the LLVM inliner.
AI disclosure: Claude wrote the code, I wrote the commit message and
have done initial review.
[AI test fix] Use property syntax for alias scope attributes
The strict-properties assembly format change means inherent attributes
such as `alignment`, `alias_scopes` and `noalias_scopes` on
`llvm.load`/`llvm.store` can no longer be parsed out of the trailing
attribute dictionary. Switch the newly added tests to the `<...>`
property syntax already used by the rest of these files.
[mlir][LLVM] Add disjointScopes to AliasScopeDomainAttr
This also updates the MLIR-side inliner to clone disjoint domains
while cloning alias scopes, matching changes to LLVM.
AI disclosure: Claude wrote the code, I wrote the commit message and
looked at the code.
[AMDGPU] Use a disjoint scope domain for merged LDS structs
When lowering LDS values, all the values are mutually disjoint, so we
can use the newly-added disjoint scopes feature to simplify the IR.
AI disclosure: Claude wrote this and I reviewed it and wrote the
commit message
[AMDGPU] Use a disjoint scope domain for noalias kernel arguments
All noalias arguments of a kernel are disjoint with each other, so we
can use a disjoint scope to save on metadata construction.
AI disclosure: Claude wrote this, I looked at it and wrote this
message.
[Inliner] Use a disjoint scope domain for noalias arguments
InlineFunction creates alias.scope/noalias metadata to represent the
set of `noalias` arguments to a function. We don't need the `!noalias`
now that we have the ability to use disjoint scopes, saving us IR size
and metadata bloat.
TODO move these to a previous commit.
Also changes InstCombine to not drop the experimental.noalias.scope.decl
for disjoint scopes even if they're not mentioned in a `!noalias`, but
do still delete them if they're not used.
[CodeGen] Report an error for a direct inline asm output in memory
An inline asm output returned by value has no memory to write to, yet a
constraint such as "=rm" picks memory, the most general constraint, as
does "=m". SelectionDAG asserted on that ("Can only indirectify direct
input operands!"), and GlobalISel dereferenced a null pointer. Clang
never emits such an output, since it passes the address of a memory
output, but other IR can. Report "cannot handle direct memory outputs
yet for constraint 'm'" instead, like the other inline asm errors there.
Assisted-by: Claude Opus 5.5
[TargetInstrInfo] Fix folding inline asm operands next to tied operands
foldInlineAsmMemOperand() swapped a register operand for the target's
memory operands with MachineInstr::removeOperand(), which asserts when a
later operand is tied, because moving it would break the tie. Inline asm
lists every input after every output, so folding any operand that comes
before another tied pair asserted, e.g. an "rm" input followed by the
input of a "+r" operand, or one of two "+rm" operands. Without
assertions, the moved operands kept stale tie indices. Untie the
operands, rebuild the operand list, and re-tie the remaining pairs at
their new positions.
It also gave up when the register appears in more than one operand, e.g.
one value passed to two "rm" operands, which left the greedy allocator
unable to spill that value at all. Fold every such operand into the
stack slot.
Finally, take MayLoad from the folded operands rather than from every
read of the register: a folded def whose tied use is another virtual
[9 lines not shown]
[MIR] Serialize the "foldable" inline asm register operand flag
The symbolic MIR syntax for inline asm operand flags dropped
InlineAsm::Flag's RegMayBeFolded bit, which marks a register operand
(from an "rm" constraint) that the register allocator may fold to a
stack slot. Printing MIR and parsing it back therefore silently changed
what the allocator was allowed to do, e.g. with -stop-before and
-run-pass, and a test could only set the bit through a raw numeric flag.
Print the bit as a trailing "foldable", as MachineInstr::print() already
does, and parse it back. A tied use stores its matched operand number in
the same bits, so it never prints the marker.
Assisted-by: Claude Opus 5.5
Remove (mostly) redundant run line
The second run line showed that the *printed* form of the numeric flag (1076101129 -> reguse:GR32 foldable) parses back. But the first run line already contains the symbolic syntax, so it may be a bit overkill.
A tied inline asm use is never foldable
A tied use stores its matched operand number in the bits that otherwise
hold RegMayBeFolded. Make InlineAsm::Flag enforce that: setRegMayBeFolded()
asserts the operand isn't tied, and getRegMayBeFolded() returns false for a
tied use. The MIR printer no longer needs to special-case tied uses, so
print 'foldable' independently of 'tiedto' and drop the comments that
explained the shared bits.
Also use F.isMemKind() instead of comparing K in the MIR parser.
Assisted-by: Claude Opus 5.5
Parse 'foldable' in one place and test the class-less form
Replace the ParseOptionalFoldable lambda with a single check at the end of
parseSymbolicInlineAsmOperand by making the ':' constraint parsing
conditional. Add class-less "regdef foldable" and "reguse foldable" operands
to inline-asm-foldable.mir, which take the path without a ':' constraint
that wasn't tested before.
Assisted-by: Claude Opus 5.5
AMDGPU: Create the implicit argument attributes once
AAAMDAttributes created all 21 string attributes again for every
function it manifested, which hashes the attribute name to find the
uniqued attribute. Create them once sorted in the information cache.
Instructions spent in amdgpu-attributor:
rocPRIM test_device_scan, gfx942: 1376M -> 1313M (-4.5%)
Blender Cycles HIP kernel, gfx1100: 503M -> 502M (-0.1%)
rocPRIM 16 tests, full LTO, closed world: 5148M -> 4982M (-3.2%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[SPIRV] Add size optimisation for AMDGCN flavoured SPIR-V (#228188)
For AMDGCN flavoured SPIR-V we disable optimisations on SPIR-V IR, to
prevent introducing changes that are detrimental to the subsequent
lowering to AMDGPU. However, this can lead to fairly voluminous
binaries, in extreme cases. This patch adds a minimal set of size
optimising passes that *should* be safe to run. They execute for
`spirv64-amd-amdhsa` codegen when passing the dedicated
`-fenable-spirv-size-optimization` flag, for optimisation levels greater
than `O0`.
[CodeGen] Drop dead SlotIndexes before allocation
SlotIndexes keeps the index list entry of an erased instruction and only
clears its instruction pointer. Live range sizes are measured in slot
indexes and greedy ranks ranges by size, so the leftovers inflate some
ranges more than others and reorder allocation, spilling heavily on
register-starved functions.
Add SlotIndexes::compactIndexes() to erase them. Erased entries are
unlinked, so LiveIntervals first reports the indexes it holds via
appendReferencedIndexes().
Off by default behind -greedy-compact-slot-indexes, since it changes
allocation across much of the test suite.
[clang][CodeGen] Add release note for EmitDeferred re-emission fix
Follow-up to #223877, which fixed a use-after-free when an already-emitted deferred declaration was emitted a second time. The release note was missed before that PR merged.
