[NFC][Attributor] Reach fixpoint in call site AACallEdges for direct calls
A direct call whose callee has no callback metadata only calls its callee, so
its call site AACallEdges cannot change after initialization. Add the edge in
initialize and reach a fixpoint there, instead of updating the attribute twice
to find out. AACallEdgesFunction already skips the call site attribute for such
calls since #232616; both now share getOnlyCallee().
Other attributes still create call site AACallEdges for direct calls, e.g.,
AAInterFnReachability and the AMDGPU attributor. On a generated module with
2,001 functions and 15,986 calls, this removes all updates of call site
AACallEdges. The total number of updates goes from 304,263 to 272,291 with
-passes=attributor, from 207,815 to 175,843 with -passes=attributor-light, and
from 35,977 to 16,005 with -passes=amdgpu-attributor. The wall time does not
change measurably on that module, because these updates are cheap.
This PR was assisted by AI but I reviewed all the changes.
[NFC][AMDGPU] Don't create call site AACallEdges for direct calls
AAAMDGPUMinAGPRAlloc and AAAMDGPUNoLDSDMA asked for a call site AACallEdges for
every call that is not an intrinsic, only to get its callees. For a direct call
whose callee has no callback metadata, that callee is the only one, so use it
directly, like AACallEdgesFunction does since #232616. Calls to callback brokers
still use AACallEdges, because thebroker also calls the callback target.
On a generated module with 2,001 functions and 15,986 calls,
-passes=amdgpu-attributor creates 2,001 instead of 11,987 abstract attributes,
and the median wall time goes from 95 ms to 84 ms.
This PR was assisted by AI but I reviewed all the changes.
[ORC] Match const void *Tag signature in orc_rt_lite_reoptimize_helper
SelfExecutorProcessControl::jitDispatchViaWrapperFunctionManager takes a const void *FnTag second parameter; using void *Tag in orc_rt_lite_reoptimize_helper's JITDispatch function pointer trips -fsanitize=cfi-icall.
Assisted-by: Gemini
[Matrix] Start fused loop FP accumulators at -0.0. (#237649)
Start FP accumulators at -0.0, the identity of fadd. The old start of
+0.0 can cause incorrect results.
PR: https://github.com/llvm/llvm-project/pull/237649
Attributor: Update the call graph for functions with replaced uses (#231546)
When the Attributor replaced the uses of a value, it only marked the
function of the replaced value as needing a call graph update if that
value was an instruction. Replacing an argument with a function, e.g., a
function pointer passed by all callers, adds a reference to that function
without updating the call graph. The next function pass in the CGSCC
pipeline then hit an assertion about new reference edges. Mark the
function of every changed use instead.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir][tblgen] Finish checked formatting migration through PDLL
Register parsed TableGen source buffers before processing imported
records. Format the predicate before wrapping it in the generated
success return, and propagate failure without appending an invalid
declaration.
Adapt shared formatting messages to PDLL diagnostics. Anchor the primary
at the import in the main document so LSP publishes it, and retain the
constraint definition, multiclass instantiations, and nested PDLL import
locations as notes. Formatting errors return through the parser and
leave the language server alive for subsequent edits.
Deprecate pointer-taking factories, lazy object types, PHKind shims, and
the unchecked summary helper while retaining compatibility behavior.
Migrate successful format tests and suppress warnings locally around
intentional legacy tests. Document recoverable import errors in the PDLL
reference.
[9 lines not shown]
[mlir][tblgen] Check substitutions throughout code generation
Migrate generator and shared-verifier emission to eager fatal formatting
with template-definition origins and operation use notes. Synthesized
properties without a record fall back to the operation location.
Speculative attribute and property uniquing now consumes checked errors
and falls back to inline verification. Keep value-specific bindings
local to their verifier.
Add checked summary-expression formatting that retains the escaped
segment and propagates its owned error to the source diagnostic
boundary. Expose reference-taking tgfmt and tgfmtv overloads returning
Expected, retain transition aliases, and document formatting semantics
and diagnostics in the existing ODS guide.
Escape literal dollars in a test builder comment, and keep the nested
matcher error test focused on its intended invalid DAG. Add regression
coverage for shared and inline constraints, summaries, attribute
templates, interface templates, and the reference-taking API.
[6 lines not shown]
[mlir][tblgen] Diagnose substitution failures at their source
Render owned formatting errors as source-aware messages, preserving
inner definition origins, use sites, and multiclass location arrays.
Consume every error payload before printing and terminating so joined
errors and their notes are not lost.
Add eager fatal wrappers for generator boundaries and use them for trait
predicates. Missing substitutions now fail in TableGen with the
template, failed occurrences, and available bindings instead of
generating bad C++. Snapshot the trait context before value verifiers
add temporary bindings, so an inline attribute check cannot accidentally
bind $_self for a trait.
Share message construction between generic logging and source
diagnostics. Use portable death-test expressions and handle invalid
success payloads defensively at the fatal-reporting boundary.
Validation: 74 TableGen unit tests and all 69 mlir-tblgen lit tests
[6 lines not shown]
[mlir][tblgen] Add checked formatting with owned substitution errors
Add tgfmtChecked and tgfmtvChecked, validating all placeholders before
rendering and returning owned FmtError payloads on failure. Retain
failed occurrences, available substitutions, and provenance for
diagnostic users.
Unify builtin and custom binding storage: addSubst("_self", ...) now
aliases withSelf(), including consistent last-registration-wins
behavior. Parse positional indices as decimal, so $010 means 10 instead
of 8 and $08 means 8 instead of reaching an unreachable path.
Unrepresentable indices become checked errors or legacy markers instead
of assertions.
Keep the legacy API lazy, including observing later context mutations.
The compatibility PHKind entry points remain until caller migration.
Validation: 66 TableGen unit tests; unchecked Expected arguments
rejected by the compiler. Include snapshot logging and typed error
[4 lines not shown]
[SelectionDAGISel] Make IsPredicateKnownToFail return a bool and take Index by reference. NFC (#237511)
This matches all of the predicate check functions it calls. This reduces
the code, but I don't expect any performance impact. The function is
likely inlined.
Also replace Table[Index-1] with Opcode.
Assisted-by: Claude
[mlir][tblgen] Finish checked formatting migration through PDLL
Register parsed TableGen source buffers before processing imported
records. Format the predicate before wrapping it in the generated
success return, and propagate failure without appending an invalid
declaration.
Adapt shared formatting messages to PDLL diagnostics. Anchor the primary
at the import in the main document so LSP publishes it, and retain the
constraint definition, multiclass instantiations, and nested PDLL import
locations as notes. Formatting errors return through the parser and
leave the language server alive for subsequent edits.
Deprecate pointer-taking factories, lazy object types, PHKind shims, and
the unchecked summary helper while retaining compatibility behavior.
Migrate successful format tests and suppress warnings locally around
intentional legacy tests. Document recoverable import errors in the PDLL
reference.
[8 lines not shown]
[mlir][tblgen] Check substitutions throughout code generation
Migrate generator and shared-verifier emission to eager fatal formatting
with template-definition origins and operation use notes. Synthesized
properties without a record fall back to the operation location.
Speculative attribute and property uniquing now consumes checked errors
and falls back to inline verification. Keep value-specific bindings
local to their verifier.
Add checked summary-expression formatting that retains the escaped
segment and propagates its owned error to the source diagnostic
boundary. Expose reference-taking tgfmt and tgfmtv overloads returning
Expected, retain transition aliases, and document formatting semantics
and diagnostics in the existing ODS guide.
Escape literal dollars in a test builder comment, and keep the nested
matcher error test focused on its intended invalid DAG. Add regression
coverage for shared and inline constraints, summaries, attribute
templates, interface templates, and the reference-taking API.
[5 lines not shown]
[LV] Erase memcheck block early if the memory checks fold to false. (#237731)
If the memory runtime checks fold to false, a dead compare may be left
behind that keeps the hoisted bound expansions alive. Later expansions
may then re-use them, and they get removed when cleaning up the dead
check block, triggering an assertion in SCEVExpanderCleaner.
Clean up the memcheck block directly after creating the checks if they
folded to false, to avoid further expansions re-using anything left over
from MemCheckExp.
[SLP]Extract copyable scalars for isomorphic external users
External users, isomorphic to the user of a node with a copyable
scalar, extract it from that node, so other trees can reuse the
vector instead of gathering lanes from different vectors.
Assisted-by: Cursor
Reviewers: bababuck, RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/234324
[SLP]Use operand signedness for forwarded demoted and operand
A demoted and with a full-width mask is replaced by its other operand.
If that operand is narrower, the cast in the final shuffle took its
signedness from known bits, sign-extending unsigned-demoted values.
Cast the operand with its own signedness, as the regular binop path
does.
Fixes #237530
Assisted-by: Cursor
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/237748
