[AMDGPU] CallInst -> CallBase for divergence analysis (#228744)
This handles non-call instructions like callbr.
The test changes in infinite-loop.ll, where there are now two return
blocks (the original one + the dummy) instead of the unified return
block is a symptom of the callbr in those functions being not marked as
divergent anymore.
This patch was hand-manufactured using Neovim.
---------
Co-authored-by: Matt Arsenault <Matthew.Arsenault at amd.com>
Passes: Replace FunctionAttrs with attributor-light when it is enabled (#231157)
attributor-light is meant to replace FunctionAttrs, but with
-attributor-enable=light both ran. Use AttributorLightCGSCCPass in
place of each PostOrderFunctionAttrsPass, and AttributorLightPass in
place of each ReversePostOrderFunctionAttrsPass, in the default,
ThinLTO and full LTO pipelines.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[Coroutines] Avoid line 0 locations for async context projections (#231221)
CoroSplit inlines a call to the async context projection function, which
usually has line zero location as it is compiler generated boilerplate.
If that location has line zero, DwarfDebug skips the projection when it
places prologue_end, so the projection ends up in the prologue. This is
the same problem that 2cf550a04041 ([DebugInfo] Force early line-zero
calls to have meaningful locations, #156850) fixed for line 0 calls:
someone entering the function at prologue_end breaks in after the call
has completed.
If the suspend point has a line 0 location, give the projection the
location of the first instruction after the suspend point with a
non-zero line. We already do this for other intrinsics here.
Assisted by claude
[SLP]Fix copyable smin/smax identity miscompile
Fix truncation of identity value for smin/smax byt doing extra analsyis
of the bitwidth and limit.
Fixes #231210
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/231243
RuntimeLibcalls: Generate an enum for runtime library names (#230780)
Emit an RTLIB::RuntimeLibrary enumerator for each distinct LibraryName,
and use it instead of a string for isLibraryAvailable.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
RuntimeLibcalls: Remove the Default* libcall lists (#230405)
Every system library now lists only LibcallLibrary members, so nothing
references DefaultRuntimeLibcallImpls. arm64ec's '#'-prefixed set was
still derived from the whole default list minus the width slices and
the Windows exclusions. Build it from the compiler-rt, libm and libc
slices instead, still omitting the calls the Windows runtime lacks.
Take AArch64's fp128 long double calls from the libm slice, and delete
the remaining default and Windows base lists.
The generated tables are unchanged.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[MemProf] Fix remapping of matching calls when assigning function clones
When a function has several calls with identical callsite metadata, they
are recorded on one context node with the extra calls in
Node->MatchingCalls. If that function is cloned by context
disambiguation, assignFunctions must remap each version's matching calls
through the function clone's CallMap, the same way it remaps the primary
Call.
Two problems in the IR path:
* AssignCallsiteCloneToFuncClone rewrote the matching calls of the
original node (Node->MatchingCalls) instead of those of the callsite
clone being assigned. The clone node (whose MatchingCalls were copied
from the original at creation) kept the original instructions, and the
original node's list was overwritten with the clone's. UpdateCalls then
updated each function version's matching calls with the other version's
callee clone, i.e. swapped.
[16 lines not shown]
Use 'trimmed' consistently for MIB call stacks shortened by MemProfUse
'Pruning' refers to dropping unimportant non-cold contexts; rename the
tests and comments accordingly.
[MemProf] Duplicate pruned (trimmed) contexts onto callee clones
A context whose call stack was trimmed at an interior frame terminates at
an interior node of the callsite context graph: its id is on that node's
callee edges but on no caller edge. Such a context is caller-agnostic and
applies to every caller of that frame.
When that node is cloned per caller edge for a different allocation,
moveEdgeToExistingCalleeClone only moves the ids found on the caller edge
being moved, so the trimmed context stays on the original node and the new
clone's path to the trimmed allocation has no context. Function assignment
then merges that path into whichever callee version is cheapest, often one
with the wrong allocation type. The outcome depends on which other
allocations happen to be cloned above the shared frame; in a large server
a 1.1 GB allocation lost its cold hint on one path this way.
Fix: when creating a new callee clone, duplicate the contexts terminating
at the old callee (callee-edge ids minus caller-edge ids) with fresh ids
onto the clone and propagate them down its callee edges to the allocation
[8 lines not shown]
Address review: keep ContextIdToAllocationNode current during merging
Make the context id -> allocation node map used by mergeClones a member
that duplicateContextIds keeps up to date, so context ids duplicated while
creating merge nodes are visible to findOtherCallersToShareMerge.
Add mergenodes-pruned.ll (mergenodes.ll plus a third allocation whose cold
context is trimmed at foo's call to baz). This also exposed a latent issue
in assignFunctions: when cloning the other callees of a caller, the
original call was derived from the callee's original node, but that node's
Call is rewritten if the original node was itself assigned to a function
clone (IR path only; index CallInfos share the same pointer across
versions). Record the original call per node up front instead.
SelectionDAG: Mark physreg defs dead on scheduler-cloned nodes
When the list scheduler clones a node to break a physreg dependency,
the copies share one SDNode, so InstrEmitter can't tell that a copy's
implicit physreg def (e.g. EFLAGS) lost its users and didn't mark it
dead. LiveVariables recomputes it later, but dead flags are most useful
to earlier optimizations.
Instead, mark the def dead at emission time if the copy has no
scheduler edge for it. This lets convertToThreeAddress form LEAs again,
and is a step toward removing LiveVariables.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[VPlan] Rework mergeBlocksIntoPredecessors (NFC) (#230460)
Clean up the unclear logic to make it clear that we're collecting
single-successors chains in the loop region.
llc: Verify MIR outputs by default (#228248)
MIR is validated by the machine verifier on read, but by default wasn't
validated on output. This differs from opt, which runs the verifier on
output unless explicitly disabled. -verify-machineinstrs is frequently
used as a much more expensive way of getting the verifier run, since
that runs between every pass.
Run the machine verifier at the end of the pipeline whenever it stops
before code emission. Add -disable-mir-output-verify to suppress this.
It is separate from -disable-verify, which still only controls verification
of the IR input. The extra verifier is skipped when the verifier already
runs after every machine pass, with -verify-machineinstrs in the legacy
pass manager or -verify-each in the new pass manager.
Some tests had to force disabling the verifier in a few tests which already
fail the verifier.
Unlike opt, the driver can't simply verify after the pass manager finishes.
[5 lines not shown]
[AMDGPU] Recognize workitem.id.x as the lane ID in isThreadID
The matchers for wave shuffles use `isThreadID`, which only recognized
the mbcnt()-based form. However,
1) This gets canonicalized to workitem-id-based checks in some cases
and
2) Masking the workitem ID (when we know it changes between waves at a
power of 2) is a valid and typical way to express the lane ID.
Therefore, amend isThreadID to catch these forms of lane ID.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[lldb] Prefer readers with detailed debug information (#218030)
### Issue:
Some ELF binaries, including NVIDIA CUDA images, contain .debug_line
without .debug_info. LLDB currently ignores these standalone line tables
because SymbolFileDWARF only recognizes .debug_line when .debug_info is
present.
While adding support in #217932, I found a second issue: symbol-file
selection compares ability masks numerically. Consequently,
SymbolFileSymtab is being selected over SymbolFileDWARF, even though
DWARF provides line information that cannot be recovered from the symbol
table.
`SymbolFile::FindPlugin` currently compares ability masks numerically.
As a result, a reader providing compile units and line tables (`0x3`)
can lose to a basic symbol-table reader providing functions and globals
(`0x14`).
[27 lines not shown]
Handle unresolved components and part designators in locators
The locator stride walk dereferenced the symbol of every part reference,
so an unresolved component such as a%missing(::2)%f crashed semantics
before its diagnostic was reported. CheckArraySection had the same
problem for a%missing(1:2) in DEPEND, AFFINITY, and the reduction
clauses, already on main. Skip unresolved names in both checks.
AFFINITY lowering cannot describe substrings or complex parts, and the
bounds helper aborts on them. Report both as not yet implemented before
generating addresses or bounds, with or without an iterator.
[clang][AArch64] Don't round parameters to multiples of 8-byte (#215264)
If parameter is explicitly rounded to 8 byte boundary, clang must emit
either zext or sext instruction to convert the value to the target type.
This is not really required, because contents of upper bits of extended
value is not specified in AArch64 PCS.
[AMDGPU][NFC] Add lane-index-from-workitem.id.x tests for wave shuffles
mbcnt()-based lane IDs are sometimes rewritten to workitem ID-based
ones, precommit tests for this.
AI disclosure: Claude wrote these tests, I wrote the commit message
and renamed the tests / trimmed down comments
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
MC: Make usesWindowsCFI a pure object-format predicate (#231087)
Whether unwind information is encoded with SEH opcodes (.seh_*, .pdata
and .xdata) is a property of the object format. Whether a function uses
that encoding depends on the exception model, but usesWindowsCFI()
conflated the two. Make it report only the format, and add
MachineFunction::usesWindowsUnwindInfo() for the combined question and
needsWinCFI() for the common case that also requires an unwind table
entry.
This moves the exception model check to the point of use, in preparation
for removing ExceptionModel from TargetOptions. The assembler now
accepts
.seh_* directives whenever the object format supports them (i.e., MC
only tools will no longer have a notion of ExceptionModel).
Co-authored-by: Claude Opus <noreply at anthropic.com>
[flang] Add fir.assert and its LLVM lowering
Introduce a non-speculatable assertion with typed guarded results and a
write effect on a non-addressable Fortran runtime resource. Verify its
message and diagnostic operands, and erase constant-true assertions.
Keep attribute and optionality queries able to inspect guarded values
without making rewriting operations transparent to the guard.
Lower assertions to a weighted branch, a noreturn runtime call on failure,
and guarded operands on success. Pack larger diagnostic payloads into an
array on the failure path and preserve source locations and device symbol
scope. Add dialect, code generation, effect, and variable-query tests.
This introduces the operation without emitting it from source lowering.
[flang-rt] Add fatal diagnostics with integer values
Add noreturn runtime entries for messages with integer diagnostic values.
Use a fixed four-value signature for small messages and a count-and-array
signature for larger payloads, keeping both callable from device code.
The array entry accepts the restricted %lld/%% format, checks argument
counts and allocation sizes, and preserves the existing source location,
crash handler, and termination behavior. Document the runtime interfaces
in the inline-check design and cover formatting and failures in unit tests.
[DWARFLinker] Key str_offsets_base and unit_type on the unit root (#219398)
Fix DWARFv5 output for a unit whose root is DW_TAG_partial_unit, as
produced by dwz. The classic and parallel linkers rewrite strings into
DW_FORM_strx but add DW_AT_str_offsets_base only to a
DW_TAG_compile_unit root, so every string in a partial unit reads back
empty. Both also write DW_UT_compile into every unit header,
contradicting a partial unit root.
Add DW_AT_str_offsets_base to the unit root whatever its tag, and emit
DW_UT_partial for a DW_TAG_partial_unit root. Its header has the same
layout as DW_UT_compile.
The test links a partial unit with both linkers, with and without
garbage collection, and checks the unit type, the attribute, the strings
and --verify. The update mode runs pass --build-accelerator, because
--no-garbage-collection alone copies the file without linking it.
Fixes #219365.
Assisted-by: Claude Code
