[Clang][CodeGen] Pass 128-bit aggregate inline asm register inputs by value
Clang returns a register output of aggregate type, such as '_Complex
double' or a 16-byte struct, by value as an integer of the same size
(i128 for both). EmitAsmInputLValue, however, loaded aggregate register
inputs by value only up to 64 bits and passed larger ones by pointer.
So '"+r"' on a '_Complex double' emitted '"=r,*0"', which SelectionDAG
rejects as an unsupported tied indirect register input, and '"r"' on a
16-byte struct emitted '"*r"', which it rejects as an indirect register
input. Read-write '"rm"' and '"g"' operands hit the same error.
Load power-of-2 aggregate register inputs of up to 128 bits by value,
as with '__int128'. Tied aggregate operands of other sizes are still
passed by pointer and rejected.
Assisted-by: Claude Opus 5.5
[GlobalISel] Fall back on tied inline asm inputs needing several registers
InlineAsmLowering ties a matching input to a single register, and
asserted that the output it matches uses only one. But an i128 output
with an "r" constraint on AArch64 uses two 64-bit registers, so '"+r"'
on an '__int128' crashed at -O0, where GlobalISel is the default. Fall
back to SelectionDAG, which ties each register, instead.
Assisted-by: Claude Opus 5.5
[lld][MachO] Support Objective-C class selector stubs
Support Objective-C class selector stubs in lld's Mach-O backend.
This consumes the class-stub symbols emitted by Clang in [ObjC] Emit class
msgSend stub calls (#186433), using the form
`_objc_msgSendClass$<selector>$_OBJC_CLASS_$_<class>`.
Class stubs load the class object into `x0`, load the selector into `x1`, and
branch to `_objc_msgSend`. Local class symbols are materialized directly with
`adrp`/`add`; dylib or dynamic class symbols are loaded indirectly.
The implementation also tracks the resolved class symbol before dead stripping
so live class stubs can retain the needed class definition without keeping dead
dylib references alive.
Preserve the effective weakness of imported classes when adding synthesized
GOT references. Ordinary weak class references remain optional; genuine
strong references and bare stubs without weak metadata remain strong. Cover
[2 lines not shown]
[lld][MachO] Fix ObjC stubs from autolinked archives (#219743)
Move ObjC stub preparation after LC_LINKER_OPTION processing so archive
members loaded through autolink can contribute `_objc_msgSend$` selector
stubs before selector references are finalized.
Otherwise, stubs discovered from autolinked archives may not get
matching `__objc_methname` and `__objc_selrefs` entries needed during
stub emission.
[lld][MachO][NFC] Prepare ObjC stubs for variable sizes (#219742)
This is a NFC to prepare for the next PRs:
- Track the total `__objc_stubs` size explicitly as entries are added,
so stub offsets no longer assume every stub has the same size.
- Rename `symbolPrefix` to `objcMsgSendStubPrefix` for clarity.
[CIR] Emit noundef attribute during calling convention lowering (#235140)
Classic CodeGen marks an argument noundef when its ABI classification is
Extend or Indirect unless -no-enable-noundef-analysis is used. CIR
codegen defers the ABI classification, so CallConvLowering now applies
this rule when it rewrites a signature or call site.
This also corrects an issue with missing signext/zeroext flags for
transparent unions that require extension.
Assisted-by: Cursor / various models
CodeGen: Remove -verify-machineinstrs from single -run-pass MIR tests
Since #228248 / 66366908, llc verifies MIR output by default. MIR
inputs were always verified so for an llc invocation running exactly
one pass with -run-pass, the -verify-machineinstrs flag does nothing
so these flags are pointless. We should more aggressively trim
-verify-machineinstrs uses in the future, but this covers the cases
that don't change behavior. This removal was performed by generated
script.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Fix folding 64-bit move imm into subreg use in SIShrinkInstructions (#237040)
foldImmediates folded the full 64-bit immediate of a move into a
subregister use, rather than the value of the used half. Extract the
immediate value of the used subregister. Global addresses were similarly
broken.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[BoundsSafety][Docs] Add an index page and upstream status for -fbounds-safety
Add BoundsSafety/index.md as the landing page for the -fbounds-safety
docs. It covers how to contribute, standardization efforts in WG14,
open-source adoption efforts, RFCs, and talks.
Add BoundsSafetyUpstreamStatus.md, which lists what is available
upstream, what is in progress, and what is planned, with links to the
relevant PRs and issues.
Leave a stub at the old BoundsSafety.html location so existing links keep
working.
Assisted-by: Opus 5.5
[ARM][MIPS] Show reloc comments in compressed instruction byte-order (#233599)
Little endian instruction encoding for Thumb/microMIPS consists of
higher order half-word followed by lower order half-word, each in
little-endian order. Relocation bits in this encoding need special
mapping logic.
Example encoding (starting MSB): AA BB CC DD
Layout in memory (starting low byte) [BB AA] [DD CC]
Also fix test cases for Thumb/microMIPS that captured the previously
broken relocation comments.
Co-authored-by: Faraz Shahbazker <faraz.shahbazker at mediatek.com>
[AMDGPU] Preserve non-canonical values in packed f64 fneg (#236405)
Select divergent v2f64 fneg with sign-bit XORs. Restrict the
V_PK_ADD_F64 pattern to canonicalize-of-fneg so bare negation preserves
signaling NaNs and all operand bits except the sign.
Add explicit fneg and canonicalize-of-fneg coverage, and update the
affected packed vector negation checks.
Fixes #236341
---------
Co-authored-by: Jay Foad <jay.foad at gmail.com>
[AMDGPU] Fix scale_offset with wrapping offsets (#238016)
On gfx1250, vector memory scales the sign-extended offset in 64 bits,
but SMEM scales the offset in 32 bits and then zero-extends it. Both
selectors set scale_offset either way, so global and flat accesses
miscompiled a 32-bit scaled index that wraps, and SMEM loads an index
whose 64-bit scaled value reaches 4 GiB.
Keep scale_offset only when nsw or known bits prove that the hardware
computes the same address. Fix the SMEM tests that expected the wrong
code.
Fixes #237993
[X86] Enable SlotIndexes compaction by default
Compacting SlotIndexes before greedy allocation drops the entries of
erased instructions, so live range sizes and instruction distances
reflect the code being allocated rather than how much was erased
before it. AMDGPU has done this since #235806; do the same for X86.
Codegen changes only through allocation: no new crashes, verifier
errors or allocation failures in the lit suites, including
EXPENSIVE_CHECKS, and a stage-2 build of LLVM, clang and lld with
-verify-machineinstrs passes its own tests. On llvm-test-suite at -O3,
spills drop 0.3% and .text 0.06%, with compile time unchanged.
Most test updates are regenerated checks. misched-matmul now inserts
20 spills instead of 24. live-debug-vars-stale-slotindexes.mir gets
one more use of %27 so that accumulate_fit still reaches the stale
index path it was written for.
Revert "[CoroSplit] Use resume-switch weights on the final-suspend de… (#238125)
…stroy branch (#237526)"
This reverts commit c797aca0865eae6328ddd4f327d905d4115f719f.
If we have some code like this that calls f (from the introduced test
case):
```cpp
for (int i = 0; i < 100; ++i) {
Task t = f();
while (!t.crt_handle.done())
t.crt_handle.resume();
}
```
We call f.destroy 100 times, and hdl is null 100% of the time, which
contradicts the branch weights synthesized by the patch (~19/81).
[AMDGPU] Return early in scale_offset wrap checks
Each wrap check now returns false itself, so the matchers no longer
need to guard the final Offset update with ScaleOffset. The update has
to stay after the checks, because the callers use Offset even when
the matcher fails. No functional change.
[BoundsSafety][Docs][NFC] Move -fbounds-safety docs into BoundsSafety/
Move BoundsSafety.md, BoundsSafetyAdoptionGuide.md and
BoundsSafetyImplPlans.md into a BoundsSafety/ directory without changing
their content, and update the references to them. A follow-up adds an
index page for the directory.
Assisted-by: Opus 5.5
[SelectionDAG] Expose the concat fold for DAG combining
Make FoldConcatVectors a SelectionDAG method shared by node construction
and DAG combining. Invoke it directly when revisiting concat operands
instead of using getNode to rerun construction-time folds.
AMDGPU: Fix folding 64-bit move imm into subreg use in SIShrinkInstructions
foldImmediates folded the full 64-bit immediate of a move into a
subregister use, rather than the value of the used half. Extract the
immediate value of the used subregister. Global addresses were similarly
broken.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Fold global address into subregister use with lo/hi relocation
Split 64-bit relocations into the 32-bit halves when applying a
subregister extract, similar to how immediates are modified when
folding. The same split is applied when 64-bit mov pseudos are
expanded.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
SystemZ: Remove kill flags on virtual registers from MIR tests (#236919)
The verifier will soon disallow these, so prepare test inputs. Delete
tests that were identical except for the kill flags.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
libunwind: Use libgcc's function context layout for sjlj
Make call_site an i32 and the __data words pointer
sized. compiler-rt's SjLj_Function_Context was not defined in the same
way as libgcc.
This was broken for all 64-bit targets. Every field was i32, so the
exception pointer in __data[0] was truncated. It just happens that no
64-bit target defaults to SJLJ exceptions except VE. Instead of fixing
that, a VE special case was introduced in the libunwind sources and
SjLjEHPrepare, which had a special TargetMachine hook for this data
size. In reality it should just be a uintptr_t which we can already
infer from the DataLayout.
This is an ABI break for 64-bit targets other than VE when using
libunwind built for SjLj: the __data, personality, lsda and jbuf
offsets move, so compiled code and libunwind must be updated
together. That configuration was already broken by the pointer
truncation and is not the default anywhere. 32-bit targets are
[4 lines not shown]
[AMDGPU] Reword zext scale_offset test comment
SelectionDAG keeps scale_offset in this test only because it narrows
the mask to the demanded bits first. GlobalISel keeps the wider mask,
whose shift can wrap. The old comment suggested that GlobalISel misses
a proof.
[AMDGPU] Only demand the exponent bits of V_CVT_SCALEF32 scale operands (#229889)
The `V_CVT_SCALEF32_*` conversions from F32/F16/BF16 to
FP8/BF8/FP6/BF6/FP4 only read the exponent field (bits [30:23]) of their
f32 scale operand. This is documented in the CDNA4/CDNA5 ISA.
Frontends that build the scale from an E8M0 byte or an fp32 value often
mask or shift it first, e.g. `and %x, 0x7f800000` or `shl (zext i8
%e8m0), 23`, and those instructions currently survive to the final ISA.
Frontends like Triton lower straight to the ROCDL intrinsics, so the
logic in ArithToAMDGPU that works around this, doesn't reach them. Those
instructions are wasted work in the scale computation of kernels. This
patch tags the down-conversion intrinsics with a new
`AMDGPUExponentOnlyScale<ScaleArg>` class and adds a searchable table
that maps each intrinsic to its scale argument
(`AMDGPU::getExponentOnlyScaleArgIdx`). In InstCombine, the scale
operand drops any `fneg`, `fabs` or `copysign`. When the scale is a
bitcast of an i32 and every user of the bitcast is one of these scale
operands, the i32 is simplified with only the exponent bits demanded. In
[3 lines not shown]
[BoundsSafety][Docs] Add an index page and upstream status for -fbounds-safety
Add BoundsSafety/index.md as the landing page for the -fbounds-safety
docs. It covers how to contribute, standardization efforts in WG14,
open-source adoption efforts, RFCs, and talks.
Add BoundsSafetyUpstreamStatus.md, which lists what is available
upstream, what is in progress, and what is planned, with links to the
relevant PRs and issues.
Leave a stub at the old BoundsSafety.html location so existing links keep
working.
Assisted-by: Opus 5.5
libunwind: Use libgcc's function context layout for sjlj
Make call_site an i32 and the __data words pointer
sized. compiler-rt's SjLj_Function_Context was not defined in the same
way as libgcc.
This was broken for all 64-bit targets. Every field was i32, so the
exception pointer in __data[0] was truncated. It just happens that no
64-bit target defaults to SJLJ exceptions except VE. Instead of fixing
that, a VE special case was introduced in the libunwind sources and
SjLjEHPrepare, which had a special TargetMachine hook for this data
size. In reality it should just be a uintptr_t which we can already
infer from the DataLayout.
This is an ABI break for 64-bit targets other than VE when using
libunwind built for SjLj: the __data, personality, lsda and jbuf
offsets move, so compiled code and libunwind must be updated
together. That configuration was already broken by the pointer
truncation and is not the default anywhere. 32-bit targets are
[4 lines not shown]
