Attributor: Update the call graph for functions with replaced uses (#231546)
When the Attributor replaced the uses of a value, it only marked the
function of the replaced value as needing a call graph update if that
value was an instruction. Replacing an argument with a function, e.g., a
function pointer passed by all callers, adds a reference to that function
without updating the call graph. The next function pass in the CGSCC
pipeline then hit an assertion about new reference edges. Mark the
function of every changed use instead.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir][tblgen] Finish checked formatting migration through PDLL
Register parsed TableGen source buffers before processing imported
records. Format the predicate before wrapping it in the generated
success return, and propagate failure without appending an invalid
declaration.
Adapt shared formatting messages to PDLL diagnostics. Anchor the primary
at the import in the main document so LSP publishes it, and retain the
constraint definition, multiclass instantiations, and nested PDLL import
locations as notes. Formatting errors return through the parser and
leave the language server alive for subsequent edits.
Deprecate pointer-taking factories, lazy object types, PHKind shims, and
the unchecked summary helper while retaining compatibility behavior.
Migrate successful format tests and suppress warnings locally around
intentional legacy tests. Document recoverable import errors in the PDLL
reference.
[9 lines not shown]
[mlir][tblgen] Check substitutions throughout code generation
Migrate generator and shared-verifier emission to eager fatal formatting
with template-definition origins and operation use notes. Synthesized
properties without a record fall back to the operation location.
Speculative attribute and property uniquing now consumes checked errors
and falls back to inline verification. Keep value-specific bindings
local to their verifier.
Add checked summary-expression formatting that retains the escaped
segment and propagates its owned error to the source diagnostic
boundary. Expose reference-taking tgfmt and tgfmtv overloads returning
Expected, retain transition aliases, and document formatting semantics
and diagnostics in the existing ODS guide.
Escape literal dollars in a test builder comment, and keep the nested
matcher error test focused on its intended invalid DAG. Add regression
coverage for shared and inline constraints, summaries, attribute
templates, interface templates, and the reference-taking API.
[6 lines not shown]
[mlir][tblgen] Diagnose substitution failures at their source
Render owned formatting errors as source-aware messages, preserving
inner definition origins, use sites, and multiclass location arrays.
Consume every error payload before printing and terminating so joined
errors and their notes are not lost.
Add eager fatal wrappers for generator boundaries and use them for trait
predicates. Missing substitutions now fail in TableGen with the
template, failed occurrences, and available bindings instead of
generating bad C++. Snapshot the trait context before value verifiers
add temporary bindings, so an inline attribute check cannot accidentally
bind $_self for a trait.
Share message construction between generic logging and source
diagnostics. Use portable death-test expressions and handle invalid
success payloads defensively at the fatal-reporting boundary.
Validation: 74 TableGen unit tests and all 69 mlir-tblgen lit tests
[6 lines not shown]
[mlir][tblgen] Add checked formatting with owned substitution errors
Add tgfmtChecked and tgfmtvChecked, validating all placeholders before
rendering and returning owned FmtError payloads on failure. Retain
failed occurrences, available substitutions, and provenance for
diagnostic users.
Unify builtin and custom binding storage: addSubst("_self", ...) now
aliases withSelf(), including consistent last-registration-wins
behavior. Parse positional indices as decimal, so $010 means 10 instead
of 8 and $08 means 8 instead of reaching an unreachable path.
Unrepresentable indices become checked errors or legacy markers instead
of assertions.
Keep the legacy API lazy, including observing later context mutations.
The compatibility PHKind entry points remain until caller migration.
Validation: 66 TableGen unit tests; unchecked Expected arguments
rejected by the compiler. Include snapshot logging and typed error
[4 lines not shown]
[SelectionDAGISel] Make IsPredicateKnownToFail return a bool and take Index by reference. NFC (#237511)
This matches all of the predicate check functions it calls. This reduces
the code, but I don't expect any performance impact. The function is
likely inlined.
Also replace Table[Index-1] with Opcode.
Assisted-by: Claude
[mlir][tblgen] Finish checked formatting migration through PDLL
Register parsed TableGen source buffers before processing imported
records. Format the predicate before wrapping it in the generated
success return, and propagate failure without appending an invalid
declaration.
Adapt shared formatting messages to PDLL diagnostics. Anchor the primary
at the import in the main document so LSP publishes it, and retain the
constraint definition, multiclass instantiations, and nested PDLL import
locations as notes. Formatting errors return through the parser and
leave the language server alive for subsequent edits.
Deprecate pointer-taking factories, lazy object types, PHKind shims, and
the unchecked summary helper while retaining compatibility behavior.
Migrate successful format tests and suppress warnings locally around
intentional legacy tests. Document recoverable import errors in the PDLL
reference.
[8 lines not shown]
[mlir][tblgen] Check substitutions throughout code generation
Migrate generator and shared-verifier emission to eager fatal formatting
with template-definition origins and operation use notes. Synthesized
properties without a record fall back to the operation location.
Speculative attribute and property uniquing now consumes checked errors
and falls back to inline verification. Keep value-specific bindings
local to their verifier.
Add checked summary-expression formatting that retains the escaped
segment and propagates its owned error to the source diagnostic
boundary. Expose reference-taking tgfmt and tgfmtv overloads returning
Expected, retain transition aliases, and document formatting semantics
and diagnostics in the existing ODS guide.
Escape literal dollars in a test builder comment, and keep the nested
matcher error test focused on its intended invalid DAG. Add regression
coverage for shared and inline constraints, summaries, attribute
templates, interface templates, and the reference-taking API.
[5 lines not shown]
[LV] Erase memcheck block early if the memory checks fold to false. (#237731)
If the memory runtime checks fold to false, a dead compare may be left
behind that keeps the hoisted bound expansions alive. Later expansions
may then re-use them, and they get removed when cleaning up the dead
check block, triggering an assertion in SCEVExpanderCleaner.
Clean up the memcheck block directly after creating the checks if they
folded to false, to avoid further expansions re-using anything left over
from MemCheckExp.
[SLP]Extract copyable scalars for isomorphic external users
External users, isomorphic to the user of a node with a copyable
scalar, extract it from that node, so other trees can reuse the
vector instead of gathering lanes from different vectors.
Assisted-by: Cursor
Reviewers: bababuck, RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/234324
[SLP]Use operand signedness for forwarded demoted and operand
A demoted and with a full-width mask is replaced by its other operand.
If that operand is narrower, the cast in the final shuffle took its
signedness from known bits, sign-extending unsigned-demoted values.
Cast the operand with its own signedness, as the regular binop path
does.
Fixes #237530
Assisted-by: Cursor
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/237748
[SLP]Limit phi reordering to lists filling one register, NFCI
The prefix-first attempt from #234997 also ran for lists of mutually
incompatible phis and for lists longer than one register, where the
sorted windows never vectorized and only rebuilt rejected trees.
Assisted-by: Cursor
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/237743
[offload][AMDGPU] Properly scan for available SDMA engines
Two reasons:
(1) satisfy the documented API requirements of
hsa_amd_memory_async_copy_on_engine which say that the "client should
use hsa_amd_memory_copy_engine_status first to get the ID availability"
(2) work around device-dependent restrictions: the ROCr sources say that
"due to a RAS issue, GFX90a can only support H2D copies on SDMA0"
This patch was hand-manufactured using Neovim.
[SelectionDAG] Canonicalize i1 MULHS/MULHU to zero in getNode (#236643)
i1 mulh operations always return zero, because the high bit of the 2-bit
product of two booleans is never set. This canonicalizes `MULHS`/`MULHU`
to zero for i1 in `SelectionDAG::getNode`, next to the existing i1
canonicalizations for `MUL` (to `AND`) and `ADD`/`SUB` (to `XOR`).
Without it, DAGCombiner folds `mulhs(x, 1)` to `sra x, 0` — the i1
constant 1 is also -1 — and returns `x` instead of 0.
Tests: `smulh_i1` in `CodeGen/X86/smulh.ll` and `umulh_i1` in
`CodeGen/X86/umulh.ll`.
Fixes #236347
Assisted-by: Claude Code
[LV] Bail out if the vector loop becomes dead late in executePlan. (#237733)
The minimum iteration check may only fold to true after
expandSCEVsToVPInstructions, after the existing check for a dead vector
loop. expandSCEVs then still expands the now-dead runtime check bounds
that are hoisted out of the outer loop in IR. SCEVExpander may re-use
values from the SCEV checks for them, which get removed together with
the dead SCEV check block, triggering an assertion in
SCEVExpanderCleaner.
Move check just before execute, to fix assertion
[AArch64] Add a Cortex-A53 multiply-accumulate forwarding test (NFC) (#237522)
Add an llvm-mca test for the Cortex-A53 scheduling model, ahead of the
changes
that follow. Cortex/A53-mac-forwarding.s runs dependent
multiply-accumulate
chains through the accumulator and through a multiplicand. This shows
the
accumulator forwarding, which the instruction tables in
Cortex/A53-neon-instructions.s and Cortex/A53-basic-instructions.s
cannot.
The checks reflect the current model, so the forwarding changes that
follow
show up as a diff to this test.
Part 1 of 4 of the split of #219478, to be merged in order.
AI tool use: prepared with the help of Claude (Anthropic), see the
[3 lines not shown]
[libc] Fix maybe unused warnings in Arm 32-bit no-FP baremetal builds (#237257)
In configurations without floating-point, some variables end up not
used, hence warnings are shown because of this.
This patch adds `[[maybe_unused]]` to the definitions affected by this.
The ultimate goal is to re-enable `-Werror` in no-FP builds.
[mlir][tblgen] Finish checked formatting migration through PDLL
Register parsed TableGen source buffers before processing imported
records. Format the predicate before wrapping it in the generated
success return, and propagate failure without appending an invalid
declaration.
Adapt shared formatting messages to PDLL diagnostics. Anchor the primary
at the import in the main document so LSP publishes it, and retain the
constraint definition, multiclass instantiations, and nested PDLL import
locations as notes. Formatting errors return through the parser and
leave the language server alive for subsequent edits.
Deprecate pointer-taking factories, lazy object types, PHKind shims, and
the unchecked summary helper while retaining compatibility behavior.
Migrate successful format tests and suppress warnings locally around
intentional legacy tests. Complete migration and diagnostic
documentation.
[8 lines not shown]
[CostModel][X86] Add AVX512BW cost for in-lane v64i8 2 src shuffles (#236618)
Without VBMI, two-source v64i8 shuffles always got the cross-lane cost
of 19,
even when the mask stays inside 128-bit lanes (e.g.
vpunpcklbw/vpunpckhbw).
Add an in-lane entry for AVX512BW that matches the AVX2 v32i8 one
(2*vpshufb + vpor) and check it before the cross-lane table.
This lets the Stockfish NNUE loop from #236617 get fully unrolled on
AVX512BW. The new LoopUnroll test covers the unroll decision, and the
first
commit adds the tests with the current costs.
Fixes #236617
[mlir][tblgen] Check substitutions throughout code generation
Migrate generator and shared-verifier emission to eager fatal formatting
with template-definition origins and operation use notes. Synthesized
properties without a record fall back to the operation location.
Speculative attribute and property uniquing now consumes checked errors
and falls back to inline verification. Keep value-specific bindings
local to their verifier.
Add checked summary-expression formatting that retains the escaped
segment and propagates its owned error to the source diagnostic
boundary. Expose reference-taking tgfmt and tgfmtv overloads returning
Expected, retain transition aliases, and document migration and parsing
corrections.
Escape literal dollars in a test builder comment, and keep the nested
matcher error test focused on its intended invalid DAG. Add regression
coverage for shared and inline constraints, summaries, attribute
templates, interface templates, and the reference-taking API.
[5 lines not shown]
[docs] Fence more LangRef IR examples as `llvm`
Change the instruction examples that use a `<result>` placeholder, and
a few other IR examples, from `text` to `llvm`. The new lexer handles
the placeholder, which renders as plain text.
Also drop `llvm/utils/pygments/README.md` and the `pygmentize` fallback
in the lexer. The docs only load it through Sphinx.
[mlir][tblgen] Diagnose substitution failures at their source
Render owned formatting errors as source-aware messages, preserving
inner definition origins, use sites, and multiclass location arrays.
Consume every error payload before printing and terminating so joined
errors and their notes are not lost.
Add eager fatal wrappers for generator boundaries and use them for trait
predicates. Missing substitutions now fail in TableGen with the
template, failed occurrences, and available bindings instead of
generating bad C++. Snapshot the trait context before value verifiers
add temporary bindings, so an inline attribute check cannot accidentally
bind $_self for a trait.
Share message construction between generic logging and source
diagnostics. Use portable death-test expressions and handle invalid
success payloads defensively at the fatal-reporting boundary.
Validation: 74 TableGen unit tests and all 69 mlir-tblgen lit tests
[5 lines not shown]
[mlir][tblgen] Add checked formatting with owned substitution errors
Add tgfmtChecked and tgfmtvChecked, validating all placeholders before
rendering and returning owned FmtError payloads on failure. Retain
failed occurrences, available substitutions, and provenance for
diagnostic users.
Unify builtin and custom binding storage: addSubst("_self", ...) now
aliases withSelf(), including consistent last-registration-wins
behavior. Parse positional indices as decimal, so $010 means 10 instead
of 8 and $08 means 8 instead of reaching an unreachable path.
Unrepresentable indices become checked errors or legacy markers instead
of assertions.
Keep the legacy API lazy, including observing later context mutations.
The compatibility PHKind entry points remain until caller migration.
Validation: 66 TableGen unit tests; unchecked Expected arguments
rejected by the compiler. Include snapshot logging and typed error
[3 lines not shown]
